Copy and staple gestures

ABSTRACT

Techniques involving gestures and other functionality are described. In one or more implementations, the techniques describe gestures that are usable to provide inputs to a computing device. A variety of different gestures are contemplated, including bimodal gestures (e.g., using more than one type of input) and single modal gestures. Additionally, the gesture techniques may be configured to leverage these different input types to increase the amount of gestures that are made available to initiate operations of a computing device.

RELATED APPLICATIONS

This application is a Continuation of, and claims benefit from, U.S.patent application Ser. No. 12/695,842 that was filed on Jan. 28, 2010,and that is incorporated herein by reference in its entirety.

BACKGROUND

The amount of functionality that is available from computing devices isever increasing, such as from mobile devices, game consoles,televisions, set-top boxes, personal computers, and so on. However,traditional techniques that were employed to interact with the computingdevices may become less efficient as the amount of functionalityincreases.

For example, inclusion of additional functions in a menu may addadditional levels to the menu as well as additional choices at each ofthe levels. Consequently, the addition of these functions in the menumay frustrate users by the sheer number of choices of functions andthereby result in decreased utilization of both the additional functionsas well as the device itself that employs the functions. Thus,traditional techniques that were used to access the functions may limitthe usefulness of the functions to a user of the computing device.

SUMMARY

Techniques involving gestures and other functionality are described. Inone or more implementations, the techniques describe gestures that areusable to provide inputs to a computing device. A variety of differentgestures are contemplated, including bimodal gestures (e.g., using morethan one type of input) and single modal gestures. Additionally, thegesture techniques may be configured to leverage these different inputtypes to increase the amount of gestures that are made available toinitiate operations of a computing device.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is described with reference to the accompanyingfigures. In the figures, the left-most digit(s) of a reference numberidentifies the figure in which the reference number first appears. Theuse of the same reference numbers in different instances in thedescription and the figures may indicate similar or identical items.

FIG. 1 is an illustration of an environment in an example implementationthat is operable to employ gesture techniques.

FIG. 2 illustrates an example system 200 showing the gesture module 104and bimodal input module 114 of FIG. 1 as being implemented using in anenvironment where multiple devices are interconnected through a centralcomputing device.

FIG. 3 is an illustration of an example implementation in which stagesof a copy gesture of FIG. 1 are shown as being input through interactionwith the computing device.

FIG. 4 is a flow diagram depicting a procedure in an exampleimplementation of a copy gesture in accordance with one or moreembodiments.

FIG. 5 is an illustration of an example implementation in which stagesof a staple gesture of FIG. 1 are shown as being input throughinteraction with the computing device.

FIG. 6 is a flow diagram depicting a procedure in an exampleimplementation of a staple gesture in accordance with one or moreembodiments.

FIG. 7 is an illustration of an example implementation in which stagesof a cut gesture of FIG. 1 are shown as being input through interactionwith the computing device.

FIG. 8 is a flow diagram depicting a procedure in an exampleimplementation of a cut gesture in accordance with one or moreembodiments.

FIG. 9 is an illustration of an example implementation in which stagesof a punch-out gesture of FIG. 1 are shown as being input throughinteraction with the computing device.

FIG. 10 is a flow diagram depicting a procedure in an exampleimplementation of a punch-out gesture in accordance with one or moreembodiments.

FIG. 11 is an illustration of an example implementation in which acombination of a cut gesture and a punch-out gesture of FIG. 1 are shownas being input in conjunction with the computing device.

FIG. 12 is an illustration of an example implementation in which stagesof a rip gesture of FIG. 1 are shown as being input through interactionwith the computing device.

FIG. 13 is a flow diagram that depicts a procedure in an exampleimplementation of a rip gesture in accordance with one or moreembodiments.

FIG. 14 is an illustration of an example implementation in which stagesof an edge gesture of FIG. 1 are shown as being input throughinteraction with the computing device to draw a line.

FIG. 15 is a flow diagram depicting a procedure in an exampleimplementation of an edge gesture in accordance with one or moreembodiments.

FIG. 16 is a flow diagram depicting a procedure in an exampleimplementation of an edge gesture in accordance with one or moreembodiments.

FIG. 17 is an illustration of an example implementation in which stagesof an edge gesture of FIG. 1 are shown as being input throughinteraction with the computing device to cut along a line.

FIG. 18 is a flow diagram that depicts a procedure in an exampleimplementation of an edge gesture to perform a cut in accordance withone or more embodiments.

FIG. 19 is an illustration of an example implementation in which stagesof a stamp gesture of FIG. 1 are shown as being input in conjunctionwith the computing device.

FIG. 20 is a flow diagram that depicts a procedure in an exampleimplementation of a stamp gesture in accordance with one or moreembodiments.

FIG. 21 is an illustration of an example implementation in which stagesof a brush gesture of FIG. 1 are shown as being input throughinteraction with the computing device.

FIG. 22 is a flow diagram that depicts a procedure in an exampleimplementation of a brush gesture in accordance with one or moreembodiments.

FIG. 23 is an illustration of an example implementation in which stagesof a carbon-copy gesture of FIG. 1 are shown as being input throughinteraction with a computing device.

FIG. 24 is an illustration of an example implementation in which stagesof a carbon-copy gesture of FIG. 1 are shown as being input inconjunction with the computing device.

FIG. 25 is a flow diagram that depicts a procedure in an exampleimplementation of a carbon-copy gesture in accordance with one or moreembodiments.

FIG. 26 is an illustration of an example implementation in which stagesof a fill gesture of FIG. 1 are shown as being input in conjunction withthe computing device.

FIG. 27 is a flow diagram that depicts a procedure in an exampleimplementation of a fill gesture in accordance with one or moreembodiments.

FIG. 28 is an illustration of an example implementation in which stagesof a cross-reference gesture of FIG. 1 are shown as being input inconjunction with the computing device.

FIG. 29 is an illustration of an example implementation in which stagesof a gesture are shown to access metadata associated with the imageusing the fill gesture of FIG. 28.

FIG. 30 is a flow diagram that depicts a procedure in an exampleimplementation of the cross-reference gesture of FIG. 1 in accordancewith one or more embodiments.

FIG. 31 is an illustration of an example implementation in which stagesof a link gesture of FIG. 1 are shown as being input in conjunction withthe computing device.

FIG. 32 is a flow diagram that depicts a procedure in an exampleimplementation of a link gesture in accordance with one or moreembodiments.

FIG. 33 is an illustration of an example implementation in which stagesof the link gesture of FIG. 1 are shown as being input in conjunctionwith the computing device.

FIG. 34 is a flow diagram that depicts a procedure in an exampleimplementation of a link gesture in accordance with one or moreembodiments.

FIG. 35 depicts an example implementation showing techniques forcontextual spatial multiplexing.

FIG. 36 is a flow diagram that depicts a procedure in an exampleimplementation in which identification of whether an input is a stylusor touch input is used to identify an operation to be performed inconjunction with a user interface.

FIG. 37 is a flow diagram that depicts another procedure in an exampleimplementation in which identification of whether an input is a stylusor touch input is used to identify an operation to be performed inconjunction with a user interface.

FIG. 38 illustrates various components of an example device that can beimplemented as any type of portable and/or computer device as describedwith reference to FIGS. 1-37 to implement embodiments of the gesturetechniques described herein.

DETAILED DESCRIPTION

Overview

Conventional techniques that were used to access functions of acomputing device may become less efficient when expanded to access anever increasing number of functions. Therefore, these conventionaltechniques may result in user frustration regarding the additionalfunctions and may result in decreased user satisfaction with a computingdevice having those additional functions. For example, use of atraditional menu may force a user to navigate through multiple levelsand selections at each of the levels to locate a desired function, whichmay be both time consuming and frustrating for the user.

Techniques involving gestures are described. In the followingdiscussion, a variety of different implementations are described thatinvolve gestures to initiate functions of a computing device. In thisway, a user may readily access the functions in an efficient andintuitive manner without encountering the complexities involved usingconventional access techniques. For example, in one or moreimplementations a gesture involves a bimodal input to signify thegesture, such as through direct manual input using touch (e.g., a fingerof a user's hand) and a stylus (e.g., a pointed input device such as apen). Through recognition of which input is a touch input versus astylus input and vice versa, a variety of different gestures may besupported. Further discussion of this and other implementations that doand do not involve bimodal inputs may be found in the followingsections.

In the following discussion, an example environment is first describedthat is operable to employ the gesture techniques described herein.Example illustrations of gestures and procedures involving the gesturesare then described, which may be employed in the example environment aswell as in other environments. Accordingly, the example environment isnot limited to performing the example gestures and procedures. Likewise,the example procedures and gestures are not limited to implementation inthe example environment.

Example Environment

FIG. 1 is an illustration of an environment 100 in an exampleimplementation that is operable to employ gesture techniques. Theillustrated environment 100 includes an example of a computing device102 that may be configured in a variety of ways. For example, thecomputing device 102 may be configured as a traditional computer (e.g.,a desktop personal computer, laptop computer, and so on), a mobilestation, an entertainment appliance, a set-top box communicativelycoupled to a television, a wireless phone, a netbook, a game console,and so forth as further described in relation to FIG. 2. Thus, thecomputing device 102 may range from full resource devices withsubstantial memory and processor resources (e.g., personal computers,game consoles) to a low-resource device with limited memory and/orprocessing resources (e.g., traditional set-top boxes, hand-held gameconsoles). The computing device 102 may also relate to software thatcauses the computing device 102 to perform one or more operations.

The computing device 102 is illustrated as including a gesture module104. The gesture module 104 is representative of functionality toidentify gestures and cause operations to be performed that correspondto the gestures. The gestures may be identified by the gesture module104 in a variety of different ways. For example, the gesture module 104may be configured to recognize a touch input, such as a finger of auser's hand 106 as proximal to a display device 108 of the computingdevice 102 using touchscreen functionality.

The touch input may also be recognized as including attributes (e.g.,movement, selection point, etc.) that are usable to differentiate thetouch input from other touch inputs recognized by the gesture module104. This differentiation may then serve as a basis to identify agesture from the touch inputs and consequently an operation that is tobe performed based on identification of the gesture.

For example, a finger of the user's hand 106 is illustrated as selecting110 an image 112 displayed by the display device 108. Selection 110 ofthe image 112 and subsequent movement of the finger of the user's hand106 may be recognized by the gesture module 104. The gesture module 104may then identify this recognized movement as indicating a “drag anddrop” operation to change a location of the image 112 to a point in thedisplay at which the finger of the user's hand 106 was lifted away fromthe display device 108. Thus, recognition of the touch input thatdescribes selection of the image, movement of the selection point toanother location, and then lifting of the finger of the user's hand 106may be used to identify a gesture (e.g., drag-and-drop gesture) that isto initiate the drag-and-drop operation.

A variety of different types of gestures may be recognized by thegesture module 104, such a gestures that are recognized from a singletype of input (e.g., touch gestures such as the previously describeddrag-and-drop gesture) as well as gestures involving multiple types ofinputs. As illustrated in FIG. 1, for instance, the gesture module 104is illustrated as including a bimodal input module 114 that isrepresentative of functionality to recognize inputs and identifygestures involving bimodal inputs.

For example, the computing device 102 may be configured to detect anddifferentiate between a touch input (e.g., provided by one or morefingers of the user's hand 106) and a stylus input (e.g., provided by astylus 116). The differentiation may be performed in a variety of ways,such as by detecting an amount of the display device 108 that iscontacted by the finger of the user's hand 106 versus an amount of thedisplay device 108 that is contacted by the stylus 116. Differentiationmay also be performed through use of a camera to distinguish a touchinput (e.g., holding up one or more fingers) from a stylus input (e.g.,holding two fingers together to indicate a point) in a natural userinterface (NUI). A variety of other example techniques fordistinguishing touch and stylus inputs are contemplated, furtherdiscussion of which may be found in relation to FIG. 38.

Thus, the gesture module 104 may support a variety of different gesturetechniques by recognizing and leveraging a division between stylus andtouch inputs through use of the bimodal input module 114. For instance,the bimodal input module 114 may be configured to recognize the stylusas a writing tool, whereas touch is employed to manipulate objectsdisplayed by the display device 108. Consequently, the combination oftouch and stylus inputs may serve as a basis to indicate a variety ofdifferent gestures. For instance, primitives of touch (e.g., tap, hold,two-finger hold, grab, cross, pinch, hand or finger postures, and so on)and stylus (e.g., tap, hold-and-drag-off, drag-into, cross, stroke) maybe composed to create a space of intuitive and semantically richgestures. It should be noted that by differentiating between stylus andtouch inputs, the number of gestures that are made possible by each ofthese inputs alone is also increased. For example, although themovements may be the same, different gestures (or different parametersto analogous commands) may be indicated using touch inputs versus stylusinputs.

Accordingly, the gesture module 104 may support a variety of differentgestures, both bimodal and otherwise. Examples of gestures describedherein include a copy gesture 118, a staple gesture 120, a cut gesture122, a punch-out gesture 124, a rip gesture 126, an edge gesture 128, astamp gesture 130, a brush gesture 132, a carbon-copy gesture 134, afill gesture 136, a cross-reference gesture 138, and a link gesture 140.Each of these different gestures is described in a corresponding sectionin the following discussion. Although different sections are used, itshould be readily apparent that the features of these gestures may becombined and/or separated to support additional gestures. Therefore, thedescription is not limited to these examples.

Additionally, although the following discussion may describe specificexamples of touch and stylus inputs, in instances the types of inputsmay be switched (e.g., touch may be used to replace stylus and viceversa) and even removed (e.g., both inputs may be provided using touchor a stylus) without departing from the spirit and scope thereof.Further, although in instances in the following discussion the gesturesare illustrated as being input using touchscreen functionality, thegestures may be input using a variety of different techniques by avariety of different devices, further discussion of which may be foundin relation to the following figure.

FIG. 2 illustrates an example system 200 showing the gesture module 104and bimodal input module 114 of FIG. 1 as being implemented using in anenvironment where multiple devices are interconnected through a centralcomputing device. The central computing device may be local to themultiple devices or may be located remotely from the multiple devices.In one embodiment, the central computing device is a “cloud” serverfarm, which comprises one or more server computers that are connected tothe multiple devices through a network or the Internet or other means.In one embodiment, this interconnection architecture enablesfunctionality to be delivered across multiple devices to provide acommon and seamless experience to the user of the multiple devices. Eachof the multiple devices may have different physical requirements andcapabilities, and the central computing device uses a platform to enablethe delivery of an experience to the device that is both tailored to thedevice and yet common to all devices. In one embodiment, a “class” oftarget device is created and experiences are tailored to the genericclass of devices. A class of device may be defined by physical featuresor usage or other common characteristics of the devices.

For example, as previously described the computing device 102 may assumea variety of different configurations, such as for mobile 202, computer204, and television 206 uses. Each of these configurations has agenerally corresponding screen size and thus the computing device 102may be configured accordingly to one or more of these device classes inthis example system 200. For instance, the computing device 102 mayassume the mobile 202 class of device which includes mobile phones,portable music players, game devices, and so on. The computing device102 may also assume a computer 204 class of device that includespersonal computers, laptop computers, netbooks, and so on. Thetelevision 206 configuration includes configurations of devices thatinvolve display on a generally larger screen in a casual environment,e.g., televisions, set-top boxes, game consoles, and so on. Thus, thetechniques described herein may be supported by these variousconfigurations of the computing device 102 and are not limited to thespecific examples described in the following sections.

The cloud 208 is illustrated as including a platform 210 for webservices 212. The platform 210 abstracts underlying functionality ofhardware (e.g., servers) and software resources of the cloud 208 andthus may act as a “cloud operating system.” For example, the platform210 may abstract resources to connect the computing device 102 withother computing devices. The platform 210 may also serve to abstractscaling of resources to provide a corresponding level of scale toencountered demand for the web services 212 that are implemented via theplatform 210. A variety of other examples are also contemplated, such asload balancing of servers in a server farm, protection against maliciousparties (e.g., spam, viruses, and other malware), and so on. Thus, webservices 212 and other functionality may be supported without thefunctionality “having to know” the particulars of the supportinghardware, software, and network resources.

Accordingly, in an interconnected device embodiment, implementation offunctionality of the gesture module 104 (and the bimodal input module114) may be distributed throughout the system 200. For example, thegesture module 104 may be implemented in part on the computing device102 as well as via the platform 210 that abstracts the functionality ofthe cloud 208.

Further, the functionality may be supported by the computing device 102regardless of the configuration. For example, the gesture techniquessupported by the gesture module 104 may be detected using touchscreenfunctionality in the mobile 202 configuration, track pad functionalityof the computer 204 configuration, detected by a camera as part ofsupport of a natural user interface (NUI) that does not involve contactwith a specific input device in the television 206 example, and so on.Further, performance of the operations to detect and recognize theinputs to identify a particular gesture may be distributed throughoutthe system 200, such as by the computing device 102 and/or the webservices 212 supported by the platform 210 of the cloud 208. Furtherdiscussion of the gestures supported by the gesture module 104 may befound in relation to the following sections.

Generally, any of the functions described herein can be implementedusing software, firmware, hardware (e.g., fixed logic circuitry), manualprocessing, or a combination of these implementations. The terms“module,” “functionality,” and “logic” as used herein generallyrepresent software, firmware, hardware, or a combination thereof. In thecase of a software implementation, the module, functionality, or logicrepresents program code that performs specified tasks when executed on aprocessor (e.g., CPU or CPUs). The program code can be stored in one ormore computer readable memory devices. The features of the gesturetechniques described below are platform-independent, meaning that thetechniques may be implemented on a variety of commercial computingplatforms having a variety of processors.

Copy Gesture

FIG. 3 is an illustration of an example implementation 300 in whichstages of the copy gesture 118 of FIG. 1 are shown as being inputthrough interaction with the computing device 102. The copy gesture 118is illustrated in FIG. 3 using first, second, and third stages 302, 304,306. In the first stage 302, an image 308 is displayed by the displaydevice 108 of the computing device 102. The image 308 is furtherillustrated as being selected 310 using a finger of the user's hand 106.For example, the finger of the user's hand 106 may be placed and heldwithin a boundary of image 308. This touch input may therefore berecognized by the gesture module 104 of the computing device 102 as atouch input to select the image 308. Although selection with the user'sfinger has been described, other touch inputs are also contemplatedwithout departing from the spirit and scope thereof.

In the second stage 304, the image 308 is still selected using thefinger of the user's hand 106, although in other embodiments the image308 may remain in a selected state even after the finger of the user'shand 106 is lifted away from the image 308. While the image 308 isselected, the stylus 116 is used to provide a stylus input that includesplacement of the stylus within the boundary of the image 308 and thensubsequent movement of the stylus outside the boundary of the image 308.This movement is illustrated in the second stage 304 using a phantomline and a circle indicating the initial point of interaction with theimage 308 by the stylus 116. Responsive to the touch and stylus inputs,the computing device 102 (through the gesture module 104) causes a copy312 of the image 308 to be displayed by the display device 108. The copy312 in this example follows the movement of the stylus 116 at theinitial point of interaction with the image 308. In other words, theinitial point of interaction of the stylus 116 with the image 308 isused as a continuing point for manipulation of the copy 312 such thatthe copy 312 follows movement of the stylus. In an implementation, thecopy 312 of the image 308 is displayed once the movement of the stylus116 crosses a boundary of the image 308, although other implementationsare also contemplated, such as movement that has passed a thresholddistance, recognition of the touch and stylus inputs as indicating thecopy gesture 118, and so on. For example, if a boundary edge of an imagelies beyond a maximum allowed stroke distance from a starting point ofthe stylus, crossing of this maximum allowed stroke distance may insteadtrigger initiation of the copy gesture. In another example, if aboundary edge of an image is closer than a minimum allowed strokedistance, movement of the stylus beyond the minimum allowed strokedistance likewise replaces crossing of the image boundary itself. In afurther example, movement velocity may be employed rather than distancethreshold, e.g. moving the pen “fast” for a copy gesture versus slow fora carbon copy gesture. In a further example, pressure at the initiationof the movement may be employed, e.g., pressing relatively “hard” on thepen for a copy gesture.

In the third stage 306, the stylus 116 is illustrated as having movedfurther away from the image 308. In the illustrated implementation, theopacity of the copy 312 increases the further the copy 312 is moved, anexample of which may be noticed by comparison of the second and thirdstages 304, 306 that is shown using grayscale. Once the stylus 116 isremoved from the display device 108, the copy 312 is displayed at thatlocation on the display device 108 as being fully opaque, e.g., is a“true copy” of the image 308. In an implementation, another copy may becreated by repeating the stylus 116 movement while the image 308 isselected, e.g., using the finger of the user's hand 106. For example, ifthe finger of the user's hand 106 remains on the image 308 (therebyselecting the image), each subsequent movement of the stylus from withinthe boundary of the image 308 to outside the boundary may cause anothercopy of the image 308 to be created. In an implementation, the copy isnot considered to be fully realized until the copy becomes fully opaque.That is, lifting the stylus (or moving the stylus back to a distanceless than the copy creation threshold) while the image remainssemi-transparent may cancel the copy operation in this implementation.

As previously described, although a specific implementation has beendescribed using touch and stylus inputs, it should be readily apparentthat a variety of other implementations are also contemplated. Forexample, the touch and stylus inputs may be switched to perform the copygesture 118, the gesture may be performed using touch or stylus inputsalone, or a physical keyboard, mouse, or bezel button may be held downin lieu of continuing a touch input on the display device, and so on. Insome embodiments, ink annotations or other objects completely orpartially overlapping, proximal to, previously selected, or otherwiseassociated with the selected image may also be considered as part of the“image” and copied as well.

FIG. 4 is a flow diagram that depicts a procedure 400 in an exampleimplementation of a copy gesture 118 in accordance with one or moreembodiments. Aspects of the procedure may be implemented in hardware,firmware, software, or a combination thereof. The procedure is shown asa set of blocks in this example that specify operations performed by oneor more devices and are not necessarily limited to the orders shown forperforming the operations by the respective blocks. In portions of thefollowing discussion, reference will be made to the environment 100 ofFIG. 1, the system 200 of FIG. 2, and the example implementation 300 ofFIG. 3.

A first input is recognized as selecting an object displayed by adisplay device (block 402). For example, a touch input provided using afinger of a user's hand 106 may be recognized by the gesture module 104as selecting an image 308 displayed by the display device 108 of thecomputing device 102.

A second input is recognized as movement from within the bounds of theobject to outside the bounds of the object, the movement recognized asoccurring while the object is selected (block 404). Continuing with theprevious example, a stylus 116 may be used to provide an inputdescribing movement from a point within the image 308 to outside aboundary of the image 308 as shown in the second stage 304 of FIG. 3.Accordingly, the gesture module 104 may recognize this movement from astylus input detected using touchscreen functionality of the displaydevice 108. In an implementation, the first and second inputs are inputand detected concurrently using the computing device 102.

A copy gesture is identified from the recognized first and secondinputs, the copy gesture effective to cause a display of a copy of theobject to follow subsequent movement of a source of the second input(block 406). Through recognition of the first and second inputs, thegesture module 104 may identify a corresponding copy gesture 118 to beindicated using those inputs. In response, the gesture module 104 maycause a copy 312 of the image 308 to be displayed by the display device108 and follow subsequent movement of the stylus 116 across the displaydevice 108. In this way, the copy 312 of the image 308 may be createdand moved in an intuitive manner. Additional copies may also be madeusing these techniques.

For example, a third input is recognized as movement from within thebounds of the object to outside the bounds of the object, the movementrecognized as occurring while the object is selected by the first input(block 408). Thus, in this example the object (e.g., the image 308) isstill selected using the finger of the user's hand 106 (or other touchinput). Another stylus input may then be received that involves movementfrom within the image 308 to outside the bounds of the image 308.Accordingly, a second copy gesture is identified from the recognizedfirst and third inputs, the copy gesture effective to cause a display ofa second copy of the object to follow subsequent movement of a source ofthe third input (block 410).

Continuing with the previous example, the second copy may follow thesubsequent movement of the stylus 116. Although this example describedcontinued selection of the image 308 using the finger of the user's hand106, selection may continue even when the selection of the object is notcontinued using the source, e.g., the finger of the user's hand. Forexample, the image 308 may be placed in a “selected state” such thatcontinued contact by the finger of the user's hand 106 is not needed tokeep the image 308 selected. Again, it should be noted that although aspecific example was described in the copy gesture 118 above using touchand stylus inputs, those inputs may be switched, a single input type(e.g., touch or stylus) may be used to provide the input, and so on.

Staple Gesture

FIG. 5 is an illustration of an example implementation 500 in whichstages of the staple gesture 120 of FIG. 1 are shown as being input inconjunction with the computing device 102. The staple gesture 120 isillustrated in FIG. 5 using first, second, and third stages 502, 504,506. In the first stage 502, first, second, third, and fourth images508, 510, 512, 514 are displayed by the display device 108 of thecomputing device 102. A user's hands are shown in phantom as selectingthe first and second images 508, 510 using touch inputs, such as by“tapping” on the images using the user's hands.

In the second stage 504, the first and second images 508, 510 are shownas being in a selected state through the use of a phantom border aroundthe images, although other techniques may also be employed. A finger ofthe user's hand 106 is further illustrated in the second stage 504 asholding the fourth image 514, such as by placing the finger of theuser's hand 106 in the proximity of the fourth image 514 and keeping itthere, e.g., for at least a predetermined amount of time.

While the fourth image 514 is being held by the finger of the user'shand 106, the stylus 116 may be used to “tap” within the bounds of thefourth image 514. Accordingly, the gesture module 104 (and the bimodalinput module 114) may identify a staple gesture 120 from these inputs,e.g., the selection of the first and second images 508, 510, the hold ofthe fourth image 514, and the tap of the fourth image 514 using thestylus 116.

Responsive to the identification of the staple gesture 120, the gesturemodule 104 may arrange the first, second, and fourth images 508, 510,514 into a collated display. For example, the first and second images508, 510 may be displayed in the order selected as beneath the heldobject (e.g., the fourth image 514) by the display device 108.Additionally, an indication 516 may be displayed to indicate that thefirst, second, and fourth images 508, 510, 514 are stapled together. Inan embodiment, the indication 516 may be removed by holding the fourthimage 514 and swiping the stylus 116 over the indication to “remove thestaple.”

This gesture may be repeated to add additional items to the collateddisplay, e.g., selection of the third image 512 and then tapping thefourth image 514 using the stylus 116 while the fourth image 514 isbeing held. In another example, a book may be formed through use of thestaple gesture 120 to collate collections of already stapled material.Further, the collated collection of objects may be manipulated as agroup, such as to resize, move, rotate, and so on, further discussion ofwhich may be found in relation to the following figure. Performing astaple gesture on top of a stack that is already stapled may toggle thestack between collated and un-collated states (with the gesture module104 remembering the original relative spatial relationship between theitems collated), may add a cover sheet or book binding (cover) to astack, and so on.

As previously described, although a specific implementation has beendescribed using touch and stylus inputs, it should be readily apparentthat a variety of other implementations are also contemplated. Forexample, the touch and stylus inputs may be switched to perform thestaple gesture 120, the gesture may be performed using touch or stylusinputs alone, and so on.

FIG. 6 is a flow diagram that depicts a procedure 600 in an exampleimplementation of a staple gesture in accordance with one or moreembodiments. Aspects of the procedure may be implemented in hardware,firmware, software, or a combination thereof. The procedure is shown asa set of blocks in this example that specify operations performed by oneor more devices and are not necessarily limited to the orders shown forperforming the operations by the respective blocks. In portions of thefollowing discussion, reference will be made to the environment 100 ofFIG. 1, the system 200 of FIG. 2, and the example implementation 500 ofFIG. 5.

A first input is recognized as selecting a first object displayed by adisplay device (block 602). The first object may be selected in avariety of ways. For example, the first image 508 may be tapped by afinger of the user's hand 106, a stylus 116, use of a cursor controldevice, and so on.

A second input is recognized as provided subsequent to the first inputand as a hold of a second object displayed by the display device (block604). A third input is also recognized as a tap of the second objectduring the hold of the second object (block 606). Continuing with theprevious example, a finger of the user's hand 106 may be placed and heldwithin the boundary of the fourth image 514 while the stylus 116 istapped within the boundary of the fourth image 514. Additionally, theseinputs may be received after selection of the first image 508, e.g.,using a touch input.

A staple gesture is identified from the first, second, and third inputs,the staple gesture effective to cause a collated display of the firstobject as beneath the second object (block 608). The gesture module 104may identify the staple gesture 120 from the first, second, and thirdinputs. Responsive to this identification, the gesture module 104 maycause one or more objects selected by the first input to be arrangedbeneath the object that is being held as described by the second input.An example of this is shown in the third stage 506 of the system 500 ofFIG. 5. In an implementation, the one or more objects that are selectedvia the first input are arranged beneath the second input in an orderthat corresponds to an order in which the one or more objects wereselected. In other words, the order of selection of the one or moreobjects is used as a basis to arrange the objects in the collateddisplay. The collated display of the objects, as stapled together, maybe leveraged in a variety of ways.

For instance, a fourth input is recognized as involving selection of thecollated display (block 610). A gesture is identified from the fourthinput that is effective to change an appearance of the collated display(block 612). For example, the gesture may involve resizing the collateddisplay, moving the collated display, rotating the collated display,minimizing the collated display, and so on. Thus, the stapled group ofobjects may be manipulated as a group in an efficient and intuitivemanner by a user.

The staple gesture may also be repeated to add additional objects to thecollated display of a stapled group of objects, further collate groupsof already collated objects, and so on. For instance, a second staplegesture is identified that is effective to cause a collated display of athird object as beneath a fourth object (block 614). A third staplegesture is then identified that is effective to cause a collated displayof the first, second, third, and fourth objects (block 616). In thisway, a user may form a “book” of objects by repeating the staple gesture120. Again, it should be noted that although a specific example wasdescribed regarding the staple gesture 120 above using touch and stylusinputs, those inputs may be switched, a single input type (e.g., touchor stylus) may be used to provide the input, and so on.

Cut Gesture

FIG. 7 is an illustration of an example implementation 700 in whichstages of the cut gesture 122 of FIG. 1 are shown as being input throughinteraction with the computing device 102. The cut gesture 122 isillustrated in FIG. 7 using first, second, and third stages 702, 704,706. In the first stage 702, an image 708 is displayed by the displaydevice 108 of the computing device 102. A finger of the user's hand 106is illustrated as selecting the image 708 in the first stage 702.

In the second stage 704, a stylus input is received that describesmovement 710 of the stylus 116 at least twice across one or moreboundaries of the image 708 while the image 708 is selected. Thismovement 708 is illustrated through use of a dashed line in the secondstage 704 that begins outside the image 708, passes through a firstboundary of the image 708, proceeds through at least a portion of theimage 708, and passes through another boundary of the image 708, therebyleaving the confines of the image 708.

Responsive to these inputs (e.g., the touch input to select the image708 and the stylus input defining the movement), the gesture module 104may identify the cut gesture 122. Accordingly, as shown in the thirdstage 706, the gesture module 104 may cause the image 708 to bedisplayed in at least two portions 712, 714 in accordance with theindicated movement 710 of the stylus 116. In an implementation, theportions are displaced slightly in the display by the gesture module 104to better indicate the cut. Although a specific implementation has beendescribed using touch and stylus inputs, it should be readily apparentthat a variety of other implementations are also contemplated. Forexample, the touch and stylus inputs may be switched to perform the cutgesture 122, the gesture may be performed using touch or stylus inputsalone, and so on.

FIG. 8 is a flow diagram that depicts a procedure 800 in an exampleimplementation of a cut gesture in accordance with one or moreembodiments. Aspects of the procedure may be implemented in hardware,firmware, software, or a combination thereof. The procedure is shown asa set of blocks in this example that specify operations performed by oneor more devices and are not necessarily limited to the orders shown forperforming the operations by the respective blocks. In portions of thefollowing discussion, reference will be made to the environment 100 ofFIG. 1, the system 200 of FIG. 2, and the example implementation 700 ofFIG. 7.

A first input is recognized as selecting an object displayed by adisplay device (block 802). For example, the image 708 may be tapped bya finger of the user's hand 106, a stylus 116, use of a cursor controldevice, and so on. In the illustrated implementation, a finger of theuser's hand 106 is illustrated as selecting the image 708.

A second input is recognized as movement at least twice across one ormore boundaries of the object, the movement recognized as occurringwhile the object is selected (block 804). The movement may be input in avariety of ways. For example, the movement 710 may involve uninterruptedcontact of the stylus 116 with the display device 108 of the computingdevice 102 that crosses boundaries (e.g., edges) of the image 708 atleast twice. Additionally, although the movement 710 is illustrated asbeginning “outside” the image 708, the movement may also begin withinthe bounds of the image 708 and then cross at least two boundaries toindicate the cut in this example. Further, the stylus movement may alsoinclude multiple strokes (e.g., overlapping) that collectively cross theboundaries. Multiple strokes drawn in this manner may be recognized bythe module as together since the hold of the image (e.g., the touchinput) clearly indicates that these strokes belong together. To realizethis, the first (partial) stroke may put the selection in a specialstate such that additional strokes are permitted without invoking othergestures (e.g., the copy gesture) until the “phrase” of multiple strokeinputs has completed.

A cut gesture is identified from the recognized first and second inputs,the cut gesture effective to cause a display of the object to appear ascut along the movement of the second input across the display of theobject (block 806). Upon identification of the cut gesture 122 by thecomputing device 102, for instance, the gesture module 104 may cause oneor more portions of the image 106 to appear displaced from an initiallocation and to have a boundary that corresponds at least partially tothe movement 710 of the stylus 116. Furthermore, the initial and finalportions of the pen stroke (outside of the image boundary) may initiallybe seen as ordinary “ink” strokes by the gesture module 104, but duringor subsequent to the cut operation, these ink traces may be removed fromthe display device so as not to leave marks as a result of performingthe cut gesture.

It should be realized that each subsequent crossing of a boundary of theobject (e.g., the image 708) may be identified as another cut gesture.Therefore, each pair of crossings of the border of the image 708 may beidentified by the gesture module 104 as a cut. In this way, multiplecuts may be performed while the image 708 is selected, e.g., while thefinger of the user's hand 106 is still placed within the image 708.Again, it should be noted that although a specific example was describedin the cut gesture 122 above using touch and stylus inputs, those inputsmay be switched, a single input type (e.g., touch or stylus) may be usedto provide the input, and so on.

Punch-out Gesture

FIG. 9 is an illustration of an example implementation 900 in whichstages of the punch-out gesture 124 of FIG. 1 are shown as being inputthrough interaction with the computing device 102. The punch-out gesture124 is illustrated in FIG. 9 using first, second, and third stages 902,904, 906. In the first stage 902, an image 908 is illustrated as beingselected using a finger of a user's hand 106, although otherimplementations are also contemplated as previously described.

While the image 908 is selected (e.g., in a selected state), a secondinput is received that approximates self-intersecting movement 910within the image 908. For example, the movement 910 is illustrated inthe second stage 904 as being input using the stylus 116. The stylusinput that describes the movement 910 in illustrated example details anoval shown through the use of dashed lines on the image 908. In animplementation, the gesture module 104 may provide such a display (e.g.,during or upon completion of the self-intersecting movement) to serve asa visual cue to the user. Additionally, the gesture module 104 may use athreshold to identify when the movement is sufficiently close toapproximate the self-intersecting movement. In an implementation, thegesture module 104 incorporates a threshold size for the movement, e.g.,to limit punch outs below the threshold size such as at a pixel level.

At the second stage 904, the gesture module 104 has recognized that themovement 910 is self intersecting. While the image 908 is still selected(e.g., the finger of the user's hand 106 remains within the image 908),another input is received that involves a tap within the selfintersecting movement 910. For example, the stylus 116 that was used todetail the self intersecting movement 910 may then be used to tap withinthe self intersecting movement, e.g., the dashed oval as illustrated inthe second stage 904. From these inputs, the gesture module 104 mayidentify the punch-out gesture 124. In another implementation, the tapmay be performed “outside” the approximated self-intersecting movementto remove that portion of the image. Thus, the “tap” may be used toindicate which portion of the image is to be retained and which is to beremoved.

Accordingly, as illustrated in the third stage 906, a portion of theimage 908 that was within the self intersecting movement 910 is punchedout (e.g., removed) from the image 908, thereby leaving a hole 912 inthe image 908. In the illustrated implementation, the portion of theimage 908 that was punched out is no longer displayed by the displaydevice 108, although other implementations are also contemplated. Forexample, the punched out portion may be minimized and displayed withinthe hole 912 in the image 908, may be displayed proximal to the image908, and so on. Subsequent taps while the image is still held (selected)may produce additional punch-outs with the same shape as the first—thusthe operation may define a paper-punch shape and the user may thenrepeatedly apply it to punch additional holes in the image, otherimages, the canvas in the background, and so forth.

As previously described, although a specific implementation has beendescribed using touch and stylus inputs, it should be readily apparentthat a variety of other implementations are also contemplated. Forexample, the touch and stylus inputs may be switched to perform thepunch-out gesture 124, the gesture may be performed using touch orstylus inputs alone, and so on.

FIG. 10 is a flow diagram that depicts a procedure 1000 in an exampleimplementation of a punch-out gesture in accordance with one or moreembodiments. Aspects of the procedure may be implemented in hardware,firmware, software, or a combination thereof. The procedure is shown asa set of blocks in this example that specify operations performed by oneor more devices and are not necessarily limited to the orders shown forperforming the operations by the respective blocks. In portions of thefollowing discussion, reference will be made to the environment 100 ofFIG. 1, the system 200 of FIG. 2, and the example implementation 900 ofFIG. 9.

A first input is recognized as selecting an object displayed by adisplay device (block 1002). For example, the image 708 may be tapped bya finger of the user's hand 106, a stylus 116, through use of a cursorcontrol device, and so on.

A second input is recognized as self-intersecting movement within theobject (block 1004). For example, the self-intersecting movement may beinput as a continuous movement that crosses itself. A variety of shapesand sizes of self-intersecting movement are contemplated and as suchthis movement is not limited to the example illustrated movement 910 ofFIG. 9. In an implementation, the second input also includes a tapwithin an area defined by the movement as previously described inrelation to FIG. 9. However, other implementations are alsocontemplated, e.g., the portion within the self-intersecting movement910 may “drop out” without the tap of the stylus 116.

A punch-out gesture is identified from the recognized first and secondinputs, the punch-out gesture effective to cause a display of the objectto appear as if the self-intersecting movement caused a hole in theobject (block 1006). Continuing with the previous example, the hole 912may be displayed by the gesture module 104 upon identification of thepunch-out gesture 124. Again, it should be noted that although aspecific example was described in which the punch-out gesture 124 wasinput using touch and stylus inputs, those inputs may be switched, asingle input type (e.g., touch or stylus) may be used to provide theinputs, and so on. Additionally, as previously described functionalityof the gestures may be combined into a single gesture, an example ofwhich is shown in the following figure.

FIG. 11 is an illustration of an example implementation 1100 in which acombination of a cut gesture 122 and a punch-out gesture 124 of FIG. 1are shown as being input in conjunction with the computing device 102.The cut and punch-out gestures 122, 124 are illustrated through use offirst and second stages 1102, 1104. In the first stage 1102, an image1106 is illustrated as being selected using a finger of a user's hand106. Movement 1108 of a stylus 116 is also illustrated through use of adash line as before. In this case, however, the movement 1108 passesthrough two boundaries of the image 1106 and is self-intersecting withinthe image 1106.

In the second stage 1104, the image 1106 is cut along the describedmovement 1108 of the stylus 116. As with the cut gesture 122, portions1110, 1112, 1114 are slightly displaced to show “where” the image 1106was cut. Additionally, a portion of the movement 1108 is identified asself-intersecting and is therefore “punched out” of the image 1106. Inthis instance, however, the portion 1110 that is punched out isdisplayed proximal to the other portions 1112, 1114 of the image 1106.It should be readily apparent that this is but one of a variety ofdifferent examples of composing of gestures and that a variety ofdifferent combinations of the gestures described herein are contemplatedwithout departing from the spirit and scope thereof.

Rip Gesture

FIG. 12 is an illustration of an example implementation 1200 in whichstages of the rip gesture 126 of FIG. 1 are shown as being input throughinteraction with the computing device 102. The rip gesture 126 isillustrated in FIG. 12 using first and second stages 1202, 1204. In thefirst stage 1202, an image 1206 is displayed by a display device 108 ofthe computing device 102. First and second fingers of the user's hand106 and first and second fingers of the user's other hand 1208 areillustrated as selecting the image 1206. For example, the first andsecond fingers of the user's hand 106 may be used to indicate a firstpoint 1210 and the first and second finger of the user's other hand 1208may be used to indicate a second point 1212.

Movement is recognized by the gesture module 104 in which the first andsecond inputs have moved away from each other. In the illustratedimplementation, this movement 1214, 1216 describes an arc much like amotion that would be used to rip physical paper. Accordingly, thegesture module 104 may identify a rip gesture 126 from these inputs.

The second stage 1204 shows a result of the rip gesture 126. In thisexample, the image 1206 is ripped to form first and second portions1218, 1220. Additionally, a rip 1222 is formed between the first andsecond points 1210, 1212 of the image that is generally perpendicular tothe described movement of the fingers of the user's hands 106, 1208 awayfrom each other. In the illustrated example, the rip 1222 is displayedas having a jagged edge which differs from the clean edge of the cutgesture 122, although a clean edge is also contemplated in otherimplementations, e.g., to rip along a perforated line in an imagedisplayed by the display device 108. As previously described, although aspecific implementation has been described using touch and stylusinputs, it should be readily apparent that a variety of otherimplementations are also contemplated. For example, the touch and stylusinputs may be switched to perform the rip gesture 126, the gesture maybe performed using touch or stylus inputs alone, and so on.

FIG. 13 is a flow diagram that depicts a procedure 1300 in an exampleimplementation of a rip gesture 126 in accordance with one or moreembodiments. Aspects of the procedure may be implemented in hardware,firmware, software, or a combination thereof. The procedure is shown asa set of blocks in this example that specify operations performed by oneor more devices and are not necessarily limited to the orders shown forperforming the operations by the respective blocks. In portions of thefollowing discussion, reference will be made to the environment 100 ofFIG. 1, the system 200 of FIG. 2, and the example implementation 1200 ofFIG. 12.

A first input is recognized as selecting a first point of an objectdisplayed by a display device (block 1302). A second input is recognizedas selecting a second point of the object (block 1304). For example, thefinger of the user's hand 106 may select the first point 1210 and afinger from the user's other hand 1208 may select the second point ofthe image 1206.

Movement is recognized of the first and second inputs as moving awayfrom each other (block 1306). For example, the movement may contain avector component that indicates that the first and second inputs (andthus sources of the first and second inputs) are moving and/or havemoved apart. Accordingly, a rip gesture is identified from therecognized first and second inputs, the rip gesture effective to cause adisplay of the object to appear as ripped between the first and secondpoints (block 1308). As shown in FIG. 12, for instance, the rip 1222 maybe formed at a general midpoint between the first and second points1210, 1212 and run perpendicular to a straight line connecting the firstand second points 1210, 1212 if so drawn. Yet again, it should be notedthat although a specific example was described in which the rip gesture126 was input using touch inputs, those inputs may be switched to stylusinput, multiple input types may be used (e.g., touch and stylus), and soon.

Edge Gesture

FIG. 14 is an illustration of an example implementation 1400 in whichstages of the edge gesture 128 of FIG. 1 are shown as being input inconjunction with the computing device 102 to draw a line. The edgegesture 128 is illustrated in FIG. 14 using first, second, and thirdstages 1402, 1404, 1406. At the first stage 1402, an image 1408 isselected using two points of contact. For example, first and secondfingers of the user's hand 106 may be used to select the image 1408,although other examples are also contemplated. By using two-points ofcontact as opposed to one, the gesture module 104 may differentiatebetween an increased number of gestures, although it should be readilyapparent that a single point of contact is also contemplated in thisexample.

In the second stage 1404, the two points of contact from the user's hand106 are used to move and rotate the image 1408 from the initial locationin the first stage 1402 to the new location illustrated in the secondstage 1404. The stylus 116 is also illustrated as moving proximal to anedge 1410 of the image 1408. Accordingly, the gesture module 104identifies the edge gesture 128 from these inputs and causes a line 1412to be displayed as shown in the third stage 1406.

In the illustrated example, the line 1412 is displayed as proximal towhere the edge 1410 of the image 1408 was located when the movement ofthe stylus 116 occurred. Thus, in this example the edge 1410 of theimage 1408 acts as a straight edge to draw a corresponding line 1412that is straight. In an implementation, the line 1412 may be continue tofollow the edge 1410 even when proceeding past the corner of the image1408. In this way, the line 1412 may be drawn as having a length that isgreater that the length of the edge 1410.

Additionally, identification of the edge gesture 128 may cause output ofan indication 1414 of where the line is to be drawn, an example of whichis shown in the second stage 1404. The gesture module 104, for instance,may output the indication 1414 to give the user an idea of where theline 1412 is to be drawn in relation to the edge 1410. In this way, theuser may adjust a location of the image 1408 to further refine where theline is to be drawn without actually drawing the line 1412. A variety ofother examples are also contemplated without departing from the spiritand scope thereof.

In an implementation, the line 1412 has different characteristicsdepending on what is to be displayed beneath the line 1412, i.e., whatthe line is to be drawn upon. The line 1412, for instance, may beconfigured for display when drawn over a background of a user interfacebut not displayed when drawn over another image. Additionally, the image1408 may be displayed as partially transparent when used as a part ofthe edge gesture 128 such that a user may view what is beneath the image1408 and thus made better aware of a context in which the line 1412 isto be drawn. Further, although the edge 1410 is illustrated as straightin this example, the edge may take a variety of configurations, e.g., aFrench curve, circle, oval, wave, following the cut, torn, orpunched-out edges from our previous example gestures, and so on. Forinstance, a user may select from a variety of preconfigured edges toperform the edge gesture 128 (such as from a menu, a template displayedin a side area of the display device 108, and so on). Accordingly, insuch configurations the line drawn proximal to the edges may followcurves and other features of the edge.

As previously described, although a specific implementation has beendescribed using touch and stylus inputs, it should be readily apparentthat a variety of other implementations are also contemplated. Forexample, the touch and stylus inputs may be switched to perform the edgegesture 128, the gesture may be performed using touch or stylus inputsalone, and so on. For example, in some embodiments where finger paintingor color-smudging are supported using touch inputs, these touch inputsalso conform to the edge(s) thus formed. Other tools, such as anairbrush, can also be snapped to the edge, to produce a hard edge alongthe constraint line, and a soft edge on the underlying surface.

FIG. 15 is a flow diagram that depicts a procedure 1500 in an exampleimplementation of the edge gesture 128 in accordance with one or moreembodiments. Aspects of the procedure may be implemented in hardware,firmware, software, or a combination thereof. The procedure is shown asa set of blocks in this example that specify operations performed by oneor more devices and are not necessarily limited to the orders shown forperforming the operations by the respective blocks. In portions of thefollowing discussion, reference will be made to the environment 100 ofFIG. 1, the system 200 of FIG. 2, and the example implementation 1400 ofFIG. 14.

A first input is recognized as selecting an object displayed by adisplay device (block 1502). As described above, the first input may berecognized as a touch input involving two points of contact on a displayof an object, e.g., the image 1408. Although referenced as “points ofcontact,” it should be readily apparent that actual contact is notrequired, e.g., the points of contact may be signified “in the air”using a natural user interface (NUI) and detected using a camera.Therefore, the points of contact may refer to an indication of an intentto indicate contact, and not limited to actual contact itself.

A second input is recognized as movement along an edge of the object,the movement recognized as occurring while the object is selected (block1504). Continuing with the previous example, a stylus input may berecognized that was input as proximal to and following a displayed edge1410 of the image 1408 using the stylus 116.

The gesture is identified from the recognized first and second inputs,the gesture effective to cause display of a line as drawn proximal tothe edge and as following movement described by the second input (block1506). The gesture module 104 may recognize the edge gesture 128 fromthese inputs. The edge gesture 128 is operable to cause a line to bedisplayed that corresponds to the recognized movement as well as tofollow subsequent movement of the stylus 116. As previously noted, theline drawn using the edge gesture 128 is not limited to a straight lineand rather may follow any desirable edge shape without departing fromthe spirit and scope thereof. Likewise multiple pen strokes may be drawnin succession along the same or different edges of a selected object.

FIG. 16 is a flow diagram that depicts a procedure 1600 in an exampleimplementation of the edge gesture 128 in accordance with one or moreembodiments. Aspects of the procedure may be implemented in hardware,firmware, software, or a combination thereof. The procedure is shown asa set of blocks in this example that specify operations performed by oneor more devices and are not necessarily limited to the orders shown forperforming the operations by the respective blocks. In portions of thefollowing discussion, reference will be made to the environment 100 ofFIG. 1, the system 200 of FIG. 2, and the example implementation 1400 ofFIG. 14.

A first input is recognized as selecting an object displayed by adisplay device using a plurality of touch inputs (block 1602). Asdescribed in relation to FIG. 14, the first input may be recognized as atouch input involving two points of contact on a display of an object,e.g., the image 1408.

A second input is recognized as describing movement of a stylus along anedge of the object, the movement recognized as occurring while theobject is selected (block 1604). In this example, the input is a type ofstylus input that is recognized as being input as proximal to andfollowing a displayed edge 1410 of the image 1408 using the stylus 116.

A gesture is identified from the recognized first and second inputs, thegesture effective to cause the edge of the object to act as a templatesuch that a line drawn proximal to the edge as indicated by the stylusinput is displayed as following the edge of the object (block 1606).Thus, in this example, the edges of the object (e.g., image 1408) act asguide to cause display of a line responsive to identification of theedge gesture 128.

FIG. 17 is an illustration of an example implementation 1700 in whichstages of the edge gesture 128 of FIG. 1 are shown as being input inconjunction with the computing device 102 to cut along a line. The edgegesture 128 is illustrated in FIG. 17 using first, second, and thirdstages 1702, 1704, 1706. At the first stage 1702, a first image 1708 isselected using two points of contact. For example, first and secondfingers of the user's hand 106 may be used to select the image 1708,although other examples are also contemplated.

In the second stage 1704, the two points of contact from the user's hand106 are used to move the first image 1708 from the initial location inthe first stage 1702 to the new location illustrated in the second stage1704 as positioned “over” a second image 1710. Additionally, the firstimage 1708 is illustrated as partially transparent (e.g., usinggrayscale) such that at least a portion of the second image 1710 that ispositioned beneath the first image 1708 may be viewed. In this way, theuser may adjust a location of the image 1708 to further refine where thecut is to occur.

The stylus 116 is illustrated as moving proximal to an edge 1712 of thefirst image 1708 along in indication 1712 of a “cut line.” Accordingly,the gesture module 104 identifies the edge gesture 128 from theseinputs, a result of which is shown in the third stage 1706. In animplementation, the object to be cut is also selected (e.g., via a tap)to indicate what is to be cut. The selection of the edge and cut/drawnobjects may be performed in any order.

As shown in the third stage 1706, the first image 1708 has been movedaway from the second image 1710, e.g., using a drag and drop gesture tomove the image 1708 back to the previous location. Additionally, thesecond image 1710 is displayed as cut into first and second portions1714, 1716 along where the edge of the first image 1708 was positionedin the second stage 1704, i.e., along the indication 1712. Thus, in thisexample the edge of the first image 1708 may be used as a template toperform the cut, rather than performing a “freehand” cut as previouslydescribed for the cut gesture 122.

In an implementation, the cut performed by the edge gesture 128 hasdifferent characteristics depending on where the cut is to be performed.For example, the cut may be used to cut objects displayed in a userinterface but not a background of the user interface. Additionally,although the edge is illustrated as straight in this example, the edgemay take a variety of configurations, e.g., a French curve, circle,oval, wave, and so on. For instance, a user may select from a variety ofpreconfigured edges to perform cut using the edge gesture 128 (such asfrom a menu, a template displayed in a side area of the display device108, and so on). Accordingly, in such configurations the cut may followcurves and other features of a corresponding edge. Likewise, a tearinggesture may be performed with a finger to create a torn edge thatfollows the template.

As previously described, although a specific implementation has beendescribed using touch and stylus inputs, it should be readily apparentthat a variety of other implementations are also contemplated. Forexample, the touch and stylus inputs may be switched to perform the edgegesture 128, the gesture may be performed using touch or stylus inputsalone, and so on.

FIG. 18 is a flow diagram that depicts a procedure 1800 in an exampleimplementation of the edge gesture 128 to perform a cut in accordancewith one or more embodiments. Aspects of the procedure may beimplemented in hardware, firmware, software, or a combination thereof.The procedure is shown as a set of blocks in this example that specifyoperations performed by one or more devices and are not necessarilylimited to the orders shown for performing the operations by therespective blocks. In portions of the following discussion, referencewill be made to the environment 100 of FIG. 1, the system 200 of FIG. 2,and the example implementation 1700 of FIG. 17.

A first input is recognized as selecting an object displayed by adisplay device (block 1802). A second input is recognized as movementalong an edge of the object, the movement recognized as occurring whilethe object is selected (block 1804). As before, a stylus input may berecognized that was input as proximal to and following a displayed edgeof the image 1708 using the stylus 116 while the image 1708 is selected,e.g., using one or more fingers of the user's hand 106.

The gesture is identified from the recognized first and second inputs,the gesture is effective to cause display of a cut as being proximal tothe edge and as following movement described by the second input (block1806). The gesture module 104 may recognize the edge gesture 128 fromthese inputs. The edge gesture 128 is operable to cause a cut to bedisplayed that corresponds to the recognized movement as well as tofollow subsequent movement of the stylus 116. For example, portions1714, 1716 of the image 1710 may be displayed as slightly displaced toshow “where” the cut occurred. As previously noted, the cut is notlimited to a straight line and rather may follow any desirable edgeshape without departing from the spirit and scope thereof.

Again, it should be noted that although the specific examples of FIGS.14-18 were described in which the edge gesture 128 was input using touchand stylus inputs, those inputs may be switched, a single input type(e.g., touch or stylus) may be used to provide the inputs, and so on.

Stamp Gesture

FIG. 19 is an illustration of an example implementation 1900 in whichstages of the stamp gesture 130 of FIG. 1 are shown as being input inconjunction with the computing device 102. The stamp gesture 130 isillustrated in FIG. 19 using first, second, and third stages 1902, 1904,1906. In the first stage 1902, an image 1908 is selected using a fingerof the user's hand 106, although other implementations are alsocontemplated, e.g., selection using multiple points of contact, a cursorcontrol device, and so on as previously described.

In the second stage 1904, the stylus 116 is used to indicate first andsecond locations 1910, 1912 in a user interface displayed by the displaydevice 108 of the computing device 102. For example, the stylus 116 maybe used to “tap” the display device 108 at those locations. In thisexample, the first and second locations 1910, 1912 are located “outside”the bounds of the image 1908. However, it should be readily apparentthat other examples are contemplated. For instance, once the firstlocation has fallen outside the bounds of the image, a “stamping phrase”may be established, and therefore subsequent taps may fall within thebounds of the image with introducing ambiguity with respect to othergestures, e.g., the staple gesture.

Responsive to these inputs, the gesture module 104 identifies the stampgesture 130 and causes first and second copies 1914, 1916 to bedisplayed at the first and second locations 1910, 1912, respectively. Inan implementation, the first and second copies 1914, 1916 of the image1908 are displayed to give an appearance that the image 1908 was usedsimilar to a rubber stamp to stamp the copies 1914, 1916 onto abackground of the user interface. A variety of techniques may be used togive the rubber stamp appearance, such as granularity, use of one ormore colors, and so on. Furthermore the stylus tap pressure and stylustilt angles (azimuth, elevation, and roll, where available) may be usedto weight the resulting ink impression, determine the stamp's imageorientation, determine a direction of smudging or blurring effects,introduce a gradient of light to dark ink in the resulting image, and soon. Likewise, for touch inputs there may also be correspondingproperties of contact area and orientation of the touch input.Additionally, successive stamp gestures 130 may be used to createsuccessively lighter copies of the image 1908 in response to successivetaps that are performed outside the boundary of the image 1908,optionally down to a minimum lightness threshold. An example of this isillustrated as the second copy 1916 of the image 1908 being displayed aslighter than the first copy 1914 of the image 1908 in the second stage1904 through the use of grayscale. Other lightening techniques are alsocontemplated, such as use of contrast, brightness, and so on. The usermay also “refresh the ink” or change the color or effect(s) produced bya stamp by engaging a color picker, color icons, effect icons, or thelike during a stamping phrase.

In the third stage 1906, the image 1908 is displayed as rotated whencompared with the image 1908 in the first and second stages 1902, 1904.Accordingly, in this example a third stamp gesture 130 causes a displayof a third copy 1918 as having an orientation that matches theorientation of the image 1908, e.g., rotated. A variety of otherexamples are also contemplated, such as to manipulate a size, color,texture, view angle, and so on of the copies 1914-1918 of the image1908. As previously described, although a specific implementation hasbeen described using touch and stylus inputs, it should be readilyapparent that a variety of other implementations are also contemplated.For example, the touch and stylus inputs may be switched to perform thestamp gesture 130 (e.g., the image 1908 may be held using the stylus 116and the locations of where the stamp is to be made using a touch input),the gesture may be performed using touch or stylus inputs alone, and soon.

FIG. 20 is a flow diagram that depicts a procedure 2000 in an exampleimplementation of a stamp gesture 130 in accordance with one or moreembodiments. Aspects of the procedure may be implemented in hardware,firmware, software, or a combination thereof. The procedure is shown asa set of blocks in this example that specify operations performed by oneor more devices and are not necessarily limited to the orders shown forperforming the operations by the respective blocks. In portions of thefollowing discussion, reference will be made to the environment 100 ofFIG. 1, the system 200 of FIG. 2, and the system 1900 of FIG. 19.

A first input is recognized as selecting an object displayed by adisplay device (block 2002). For example, the image 1908 may be selectedusing one or more fingers of the user's hand 106, a stylus 116, use of acursor control device, and so on. Accordingly, the first input describesthis selection.

A second input is recognized as indicating a first location in a userinterface outside a boundary of the object and as occurring while theobject is selected (block 2004). For example, the second input may berecognized by the gesture module 104 as a stylus input that describes atap of the stylus 116 in the first location 1910 in a user interfacedisplayed by a display device 108 of the computing device 102.Additionally, the first location may occur outside a boundary of theimage 1908.

A first stamp gesture is identified from the recognized first and secondinputs, the first stamp gesture effective to cause a display of a copyof the object at the first location in the user interface (block 2006).Continuing with the previous example, the gesture module 104 may cause afirst copy 1914 of the image 1908 to be displayed at the first location1910. The copy 1914 of the image 1908 may be configured in a variety ofdifferent ways, such as to appear as if the image 1908 was used as arubber stamp to create the copy 1914.

Additionally, the stamp may be initiated and placed in the userinterface in a variety of ways. For example, the stylus 116 may “tapdown” on the display device 108 to indicate an initially desiredlocation, e.g., the second location 1912. If the stylus 116 is movedwhile still indicating desired interaction with the user interface(e.g., placed proximal to a user interface output by the display device108), the second copy 1916 may follow the movement of the stylus 116.Once the stylus 116 indicates a final placement, e.g., through liftingthe stylus 116 away from the display device 108, the copy may remain atthat location, motion blur/smudging may be applied to the resultingstamp that follows the path prescribed by the stylus, and so on.Additional copies (e.g., stamps) may also be made, an example of whichis described as follows.

A third input is recognized as indicating a second location in the userinterface outside the boundary of the object and as occurring while theobject is selected (block 2008). A second stamp gesture is identifiedfrom the recognized first and third inputs, the second stamp gestureeffective to cause a display of the second copy of the object at thesecond location in the user interface, the second copy being lighterthan the first copy (block 2010). Continuing yet again with the previousexample, the gesture module 104 may cause a second copy 1916 of theimage 1908 to be displayed at the second location 1912. In animplementation, successive implementation of the stamp gesture 130causes successfully lighter copies to be displayed by the display device108, an example of which is shown using successively lighter shades ofgray in the example implementation 1900 of FIG. 19. Additionally,different semantics may be employed by the gesture module 104 dependingon “what” is going to be stamped. For instance, the gesture module 104may permit the copy (e.g., the stamp) to appear on a background but noton icons or other images displayed by the display device 108, may limitthe copies to being implemented in data that may be manipulated by auser, and so on.

For example, in an embodiment an icon from a toolbar may be selected(e.g., held) and then instances of the icon may be “stamped” onto a userinterface, e.g., shapes in a drawing program. A variety of otherexamples are also contemplated. Again, it should be noted that althougha specific example was described in which the stamp gesture 130 wasinput using touch and stylus inputs, those inputs may be switched, asingle input type (e.g., touch or stylus) may be used to provide theinputs, and so on.

Brush Gesture

FIG. 21 is an illustration of an example implementation 2100 in whichstages of the brush gesture 132 of FIG. 1 are shown as being inputthrough interaction with the computing device 102. The brush gesture 132is illustrated in FIG. 21 using first, second, and third stages 2102,2104, 2106. In the first stage 2102, an image 2108 is displayed in auser interface by the display device 108 of the computing device 102.The image 2108 in this example is a photo of a city skyline having aplurality of buildings.

In the second stage 2104, a touch input is used to both select the image2108 and to select a particular point 2110 in the image 2108, which isillustrated as being performed using a finger of the user's hand 106.The stylus 116 in this example is also illustrated as providing a stylusinput that describes one or more lines that are “brushed” by the stylus116 outside the borders of the image 2108. For example, the stylus 116may make a series of zigzagging lines that begin at a location 2112outside the boundary of the image 2108 in the user interface, acombination of lines taken together, a single line over a thresholddistance, and so on. The gesture module 104 may then identify theseinputs as the brush gesture 132. At this point, the gesture module 104may consider these inputs to have initiated a brushing phrase, such thatsubsequent lines below a threshold distance are permitted.

Upon identification of the brush gesture 132, the gesture module 104 mayuse a bitmap of the image 2108 as fill for the lines drawn by the stylus116. Additionally, in an implementation the fill is taken fromcorresponding lines of the image 2108 beginning at the particular point2110 in the image 2108 indicated by the touch input, e.g., the finger ofthe user's hand 106, although other viewport mappings of the sourceimage to the resulting brush strokes are contemplated within the scopethereof, such as by using properties of the source object, e.g., textureand so on. A result of these lines is illustrated as a portion 2114 ofthe image 2108 copied using the brush strokes of the stylus 116.

In an implementation, the opacity of the lines that are drawn by thestylus 116 increases as additional lines are drawn over a given area. Asillustrated in the third stage 2106, for instance, the stylus 116 maydraw back over the portion 2114 copied from the image 2108 to increasethe opacity of the portion 2114. This is illustrated in the third stage2106 by increasing the darkness of the portion 2114 as compared to thedarkness of the portion 2114 illustrated in the second stage 2104 of theexample implementation 2100.

As previously described, although a specific implementation has beendescribed using touch and stylus inputs, it should be readily apparentthat a variety of other implementations are also contemplated. Forexample, the touch and stylus inputs may be switched to perform thebrush gesture 132, the brush gesture 132 may be performed using touch orstylus inputs alone, and so on.

FIG. 22 is a flow diagram that depicts a procedure 2200 in an exampleimplementation of a brush gesture 132 in accordance with one or moreembodiments. Aspects of the procedure may be implemented in hardware,firmware, software, or a combination thereof. The procedure is shown asa set of blocks in this example that specify operations performed by oneor more devices and are not necessarily limited to the orders shown forperforming the operations by the respective blocks. In portions of thefollowing discussion, reference will be made to the environment 100 ofFIG. 1, the system 200 of FIG. 2, and the example implementation 2100 ofFIG. 21.

A first input is recognized as selecting an object displayed by adisplay device (block 2202). For example, the image 2108 may be selectedusing a touch input, a stylus input, through use of a cursor controldevice, and so on. In the illustrated implementation, a finger of theuser's hand 106 is illustrated as selecting the image 2108 to providethe touch input.

A second input is recognized as a line drawn outside the bounds of theobject, the line recognized as being drawn while the object is selected(block 2204). For example, the second input may be a stylus input thatdescribes one or more lines drawn outside the boundary of the image 2108in a user interface.

A brush gesture is identified from the recognized first and secondinputs, the brush gesture effective to cause a display of the line beingdrawn as a copy of a corresponding line of the object (block 2206).Continuing with the previous example, the gesture module 104 mayidentify the brush gesture from the inputs and therefore use the image2108 that was selected via the first input as fill for the linesdescribed by the second input. For instance, the brush gesture may beeffective to cause a copy of a corresponding line of the objectbeginning at a point in the object selected by the first input (block2208). As shown in the second stage 2104 of FIG. 21, the touch input mayselect a particular point 2110 which may be used as a starting point toprovide fill for the lines drawn by the stylus beginning at a point 2112that is outside the image 2108. Although indication of a starting pointfor fill to be used for the brush gesture 132 by the touch input wasdescribed, a variety of other implementations are also contemplated. Forexample, the fill point for each brush gesture 132 may be set at apredefined location in the image 2108, such as a top-left corner of theimage 2108, center of the image 2108, and so on.

Additionally, the brush gesture may be effective to cause copying of acorresponding plurality of lines from the object as having a matchingspatial relationship to a plurality of lines of the second input (block2210). In this example, the lines described by the stylus input aretaken from corresponding portions of the image and preserve a spatialrelationship of the image 2108. Further, continued selection of theimage 2108 may cause lines drawn elsewhere in the user interfacedisplayed by the display device 108 to preserve this relationship untilan input is received that this is no longer desired, such as by liftinga finger of the user's hand 106 away from the display device. Therefore,even if the stylus 116 is lifted from the display device 108 and placedelsewhere on the device 108 to draw additional lines, the fill used forthose additional lines in this embodiment maintains the same spatialrelation from the image 2108 as the previous set of lines. A variety ofother examples are also contemplated, such as to begin the fill processagain using the point 2110 indicated by the touch input as a startingpoint. Again, it should be noted that although a specific example wasdescribed in which the brush gesture 132 was input using touch andstylus inputs, those inputs may be switched, a single input type (e.g.,touch or stylus) may be used to provide the inputs, and so on.

Carbon-Copy Gesture

FIG. 23 is an illustration of an example implementation 2300 in whichstages of the carbon-copy gesture 134 of FIG. 1 are shown as being inputthrough interaction with the computing device 102. The carbon-copygesture 134 is illustrated in FIG. 23 using first, second, and thirdstages 2302, 2304, 2306. In the first stage 2302, an image 2308 isdisplayed in a user interface by the display device 108 of the computingdevice 102. Like the image 2108 of FIG. 21, the image 2308 in thisexample is a photo of a city skyline having a plurality of buildings.The image 2308 is selected in the first stage 2302 using a touch input,e.g., a finger of the user's hand 106, and moved to a new location inthe user interface as illustrated in the second stage 2304.

In the second stage 2304, the stylus 116 in this example is alsoillustrated as providing a stylus input that describes one or more linesthat are “rubbed” by the stylus 116 within the borders of the image2308. For example, the stylus 116 may make a series of zigzagging linesthat begin at a location 2310 within the boundary of the image 2308 inthe user interface, a single line above a threshold length may be used,and so on as previously described. The gesture module 104 may thenidentify these inputs (e.g., the selection and rubbing) as thecarbon-copy gesture 134.

Upon identification of the carbon-copy gesture 134, the gesture module104 may use a bitmap of the image 2308, texture of the image, and so onas fill for the lines drawn by the stylus 116. Additionally, these linesmay be implemented to be drawn “through” the image 2308 such that thelines are displayed beneath the image 2308. Therefore, once the image2308 is moved away as shown in the third stage 2306, a portion 2312 ofthe image 2308 that is copied to the user interface is shown, e.g.,drawn on a background of the user interface. In an implementation, theoverlying image may be displayed as semi-transparent state to allow auser to see both the overlying and underlying images. Thus, like thebrush gesture 132, the carbon-copy gesture 134 may be used to copyportions of the image 2308 indicated by the lines drawn by the stylus116. Likewise, the image 2308 may be used as fill for the portion 2312in a variety of ways, such as use as a bitmap to make a “true” copy, useof one or more colors that may be user specified, and so on. Althoughthis example implementation 2400 showed the carbon-copy gesture 134 asbeing implemented to “deposit” the portion 2312 onto a background of auser interface, the carbon-copy gesture 134 may also be implemented to“rub up” a portion of the image 2308, an example of which is shown inthe next figure.

FIG. 24 is an illustration of an example implementation 2400 in whichstages of the carbon-copy gesture 134 of FIG. 1 are shown as being inputin conjunction with the computing device 102. Like FIG. 23, thecarbon-copy gesture 134 is illustrated in FIG. 24 using first, second,and third stages 2402, 2404, 2406. In the first stage 2402, an image2408 is displayed in a user interface by the display device 108 of thecomputing device 102. Additionally, another object 2410 is alsoillustrated in the user interface, which in this instance is shown as ablank document for clarity of the discussion, although other objects arealso contemplated. The object 2410 is selected in the first stage 2402using a touch input, e.g., a finger of the user's hand 106, and moved toa new location in the user interface (as illustrated in the second stage2404) as positioned over the image 2408, such as through use of a dragand drop gesture.

In the second stage 2404, the stylus 116 in this example is illustratedas providing a stylus input that describes one or more lines that are“rubbed” by the stylus 116 within the borders of the object 2410 and theimage 2408. For example, the stylus 116 may make a series of zigzagginglines that begin at a location within the boundary of the object 2410that is over the image 2408 in the user interface. The gesture module104 may then identify these inputs (e.g., the selection, position of theobject 2410 in relation to the image 2408, and rubbing) as thecarbon-copy gesture 134.

Upon identification of the carbon-copy gesture 134, the gesture module104 may use a bitmap of the image 2408 as fill for the lines drawn bythe stylus 116. Additionally, these lines may be implemented to be“rubbed through” onto the object 2410 such that the lines are displayedas a portion 2412 within the object 2410. Therefore, once the object2410 is moved away as shown in the third stage 2406, the portion 2412 ofthe image 2408 remains with the object 2410. Thus, like the brushgesture 132 in the previous example implementation 2300 of thecarbon-copy gesture 134, the carbon copy gesture 134 of this exampleimplementation 2400 may be used to copy portions of the image 2408indicated by the lines drawn using a stylus 116. Likewise, the image2408 may be used as fill for the portion 2412 in a variety of ways, suchas use as a bitmap to make a “true” copy, use of one or more colors thatmay be user specified, and so on.

As previously described, although a specific implementation has beendescribed using touch and stylus inputs, it should be readily apparentthat a variety of other implementations are also contemplated. Forexample, the touch and stylus inputs may be switched to perform thecarbon-copy gesture 134, the gesture may be performed using touch orstylus inputs alone, and so on.

FIG. 25 is a flow diagram that depicts a procedure 2500 in an exampleimplementation of a carbon-copy gesture 134 in accordance with one ormore embodiments. Aspects of the procedure may be implemented inhardware, firmware, software, or a combination thereof. The procedure isshown as a set of blocks in this example that specify operationsperformed by one or more devices and are not necessarily limited to theorders shown for performing the operations by the respective blocks. Inportions of the following discussion, reference will be made to theenvironment 100 of FIG. 1, the system 200 of FIG. 2, and the exampleimplementations 2300, 2400 of FIGS. 23 and 24, respectively.

A first input is recognized as selecting an object displayed by adisplay device (block 2502). For example, the image 2308 may be tappedby a finger of the user's hand 106, a stylus 116, through use of acursor control device, and so on. In the illustrated implementation ofFIG. 23, a finger of the user's hand 106 is illustrated as selecting theimage 2408. In the illustrated implementation of FIG. 24, the image 2408is selected through positioning of the object 2410 “over” the image 2408using a touch input. A variety of other examples are also contemplated.

A second input is recognized as a line that is drawn while the object isselected (block 2504). The second input, for instance, may describe aline drawn outside the bounds of the object as shown in FIG. 23. Inanother instance, the second input may describe a line drawn within thebounds of the object as shown in FIG. 24.

A carbon-copy gesture is identified from the recognized first and secondinputs, the carbon-copy gesture effect to cause display of a copy ofportions of the object (block 2506). Continuing with the previousinstances, the carbon copy gesture 134 may work to deposit portions ofthe object 2308 as shown in FIG. 23 or receive portions of the object2408 onto another object 2410 as shown in FIG. 24. It should be notedthat although a specific example was described in which the carbon-copygesture 134 was input using touch and stylus inputs, those inputs may beswitched, a single input type (e.g., touch or stylus) may be used toprovide the inputs, and so on.

Fill Gesture

FIG. 26 is an illustration of an example implementation 2600 in whichstages of the fill gesture 136 of FIG. 1 are shown as being input inconjunction with the computing device 102. The fill gesture 136 isillustrated in FIG. 26 using first, second, and third stages 2602, 2604,2606. In the first stage 2602, an image 2608 is displayed in a userinterface by the display device 108 of the computing device 102, whichmay be performed by one or more of the ways previously or subsequentlydescribed.

In the second stage 2604, a frame 2612 is illustrated as being drawnusing the stylus 116 having a rectangular shape that is defined throughmotion 2614 of the stylus 116. For example, the stylus 116 may be placedagainst the display device 108 and dragged to form the frame 2612.Although a frame 2612 having a rectangular shape is shown, a variety ofdifferent shapes may be employed as well as a variety of techniquesemployed to make them, e.g., circular, freehand, and so on.

The fill gesture 136 is then recognized from the inputs, an example of aresult of which is shown in the third stage 2606. Upon identification ofthe fill gesture 136, the gesture module 104 may use the selected image2608 to fill the frame 2612, thereby forming another image 2616. Thefill may be provided in a variety of ways, such as stretched to fit theaspect ratio of the frame 2612 as illustrated in the third stage 2606,repeated in the original aspect ratio until the frame 2612 is filled,repeated in the original aspect ratio but cropped to fit, and so on.Although a specific implementation has been described using touch andstylus inputs, it should be readily apparent that a variety of otherimplementations are also contemplated. For example, the touch and stylusinputs may be switched to perform the fill gesture 136, the fill gesture136 may be performed using touch or stylus inputs alone, and so on.

FIG. 27 is a flow diagram that depicts a procedure 2700 in an exampleimplementation of a fill gesture in accordance with one or moreembodiments. Aspects of the procedure may be implemented in hardware,firmware, software, or a combination thereof. The procedure is shown asa set of blocks in this example that specify operations performed by oneor more devices and are not necessarily limited to the orders shown forperforming the operations by the respective blocks. In portions of thefollowing discussion, reference will be made to the environment 100 ofFIG. 1, the system 200 of FIG. 2, and the example implementation 2600 ofFIG. 26.

A first input is recognized as selecting an object displayed by adisplay device (block 2702). A second input is recognized as a framedrawn outside a boundary of the object, the frame recognized as beingdrawn while the object is selected (block 2704). The frame may be drawnin a variety of ways, such as freehand using the stylus 116 or touchinput to form a self-intersecting line, selection of a preconfiguredframe, through drag-and-drop to specify a size for the frame, and so on.

A fill gesture is identified from the first and second inputs, the fillgesture effective to use the object for fill in the frame (block 2706).Upon identification of the fill gesture 136, the gesture module 104 mayuse the object selected using the first input to fill the framerecognized from the second input. The fill may be performed in a varietyof ways, such as a stretch to fill an aspect ratio of the frame 2612,repetition of the image 2608 within the frame 2612, shrinking the image2608, use of the image 2608 as a bitmap, and so on. Further, it shouldbe noted that although a specific example was described in which thefill gesture 136 was input using touch and stylus inputs, those inputsmay be switched, a single input type (e.g., touch or stylus) may be usedto provide the inputs, and so on.

Cross-reference Gesture

FIG. 28 is an illustration of an example implementation 2800 in whichstages of the cross-reference gesture 138 of FIG. 1 are shown as beinginput in conjunction with the computing device 102. The cross-referencegesture 138 is illustrated in FIG. 28 as showing the computing device102 of FIG. 1 in greater detail.

The display device 108 is illustrated as displaying an image 2802. Afinger of a user's hand 2804 is also illustrated as selecting the image2802, although as previously described a variety of different techniquesmay be used to select the image 2802.

While the image 2802 is selected (e.g., in a selected state), a stylus116 is illustrated as providing a stylus input involving one or morelines 2806, which in this instance is illustrated as the word “Eleanor.”The gesture module 104 may recognize the cross-reference gesture 138from these inputs to provide a variety of functionality.

For example, the gesture module 104 may use the cross-reference gesture138 to link the lines 2806 with the image 2802. Therefore, an operationthat causes the image 2802 to be displayed may also cause the lines 2806to be displayed in conjunction. In another example, the linkingconfigures the lines 2806 to be selectable to navigate to the image2802. For instance, selection of the lines 2806 may cause the image 2802to be displayed, a portion of a document that contains the image 2802(e.g., to jump to a page in the document that contains the image 2802),and so on. Likewise the cross-reference gesture may be used to group theobjects, such that the objects move together during drag operations, ormaintain the relative spatial relationships between image andannotations during document-reflow or other automatic or manual layoutchanges.

In a further example, the gesture module 104 may employ an ink analysisengine 2808 to identify “what is written” by the lines 2806, e.g.,convert the lines to text. For instance, the ink analysis engine 2808may be used to translate the lines 2806 to text that spells “Eleanor.”Additionally, the ink analysis engine may be used to group separatelines to be converted to text, e.g., lines that form separate charactersmay be grouped together for translation. In an implementation, the oneor more lines may provide a hint for parsing by the ink analysis engine2808, such as a special symbol to indicate that the lines are to beconverted to text.

Accordingly, the gesture module 104 through performance of thecross-reference gesture 138 may use this text in a variety of differentways. In an implementation, the text is used as a caption for theselected image 2802 and/or other metadata that may be associated withthe image, such as to identify one or more persons in the image 2802,signify a location shown in the image 2802, and so on. This metadata(e.g., the text) that is linked to the image 2802 may be accessed andleveraged for search or other tasks, an example of which is shown in thefollowing figure.

FIG. 29 is an illustration of an example implementation 2900 in whichstages of a cross-reference gesture 138 are shown to access metadataassociated with the image 2802 using the fill gesture 136 of FIG. 28.The gesture is illustrated in FIG. 29 using first, second, and thirdstages 2902, 2904, 2906. In the first stage 2902, the image 2802 of FIG.28 is displayed in a user interface by the display device 108 of thecomputing device 102. The image 2802 optionally includes an indication2908 that additional metadata is available for viewing that isassociated with the image 2802.

In the second stage 2904, a finger of the user's hand 2804 isillustrated as selecting the indication 2908 and indicating movement2910 similar to “flipping” the image 2802. In an implementation, uponidentification of these inputs the gesture module 104 may provide ananimation to give the appearance that the image 2802 is being “flippedover.” Alternatively, metadata may be revealed through a context menucommand associated with the item, e.g. a “Properties . . . ” command.

In the third stage 2906, a result of the flip gesture is shown. In thisexample, a “back” 2912 of the image 2802 is displayed. The back 2912includes a display of metadata associated with the image 2802, such aswhen the image 2802 was taken, a type for the image 2802, and themetadata input using the cross-reference gesture 138 of FIG. 28, whichis “Eleanor” in this example. The back 2912 of the image 2802 alsoincludes an indication 2914 that the back 2912 may be “flipped back” toreturn to the image 2802 shown in the first stage 2902. Although“flipping” of the image 2802 using a flip gesture was described inrelation to FIG. 29, it should be readily apparent that a variety ofdifferent techniques may be used to access the metadata.

As previously described, although a specific implementation has beendescribed using touch and/or stylus inputs in relation to FIGS. 28 and29, it should be readily apparent that a variety of otherimplementations are also contemplated. For example, the touch and stylusinputs may be switched, the gesture may be performed using touch orstylus inputs alone, and so on.

FIG. 30 is a flow diagram that depicts a procedure 3000 in an exampleimplementation of the cross-reference gesture 138 of FIG. 1 inaccordance with one or more embodiments. Aspects of the procedure may beimplemented in hardware, firmware, software, or a combination thereof.The procedure is shown as a set of blocks in this example that specifyoperations performed by one or more devices and are not necessarilylimited to the orders shown for performing the operations by therespective blocks. In portions of the following discussion, referencewill be made to the environment 100 of FIG. 1, the system 200 of FIG. 2,and the example implementations 2800, 2900 of FIGS. 28 and 29,respectively.

A first input is recognized as selecting an object displayed by adisplay device (block 3002). For example, the image 2802 may be tappedby a finger of the user's hand 2804, a stylus 116, through use of acursor control device, and so on. In the illustrated implementation, afinger of the user's hand 2804 is illustrated as selecting and holdingthe image 2802.

A second input is recognized as one or more lines drawn outside thebounds of the object, the one or more lines recognized as being drawnwhile the object is selected (block 3004). For example, the gesturemodule 104 may recognizes the lines 2806 as stylus inputs that weredrawn by the stylus 116 while the image 2802 was selected. Additionally,it should be realized that the lines 2806 may be continuous and/or madeup of segments without departing from the spirit and scope thereof.

A cross-reference gesture is identified from the recognized first andsecond inputs, the cross-reference gesture effective to cause the one ormore lines to be linked to the object (block 3006). As previouslydescribed, the lines 2806 may be linked in a variety of ways. Forexample, the gesture module 104 may employ an ink analysis engine 2808to translate the lines to text. The text may then be saved inconjunction with the image 2802, used as a link to the image 2802,displayed as a caption for the image 2802, and so on.

Again, it should be noted that although a specific example was describedin which the cross-reference gesture 138 was input using touch andstylus inputs, those inputs may be switched, a single input type (e.g.,touch or stylus) may be used to provide the inputs, and so on.

Link Gesture

FIG. 31 is an illustration of an example implementation 3100 in whichstages of the link gesture 140 of FIG. 1 are shown as being input inconjunction with the computing device 102. The link gesture 140 isillustrated in FIG. 31 using first, second, and third stages 3102, 3104,3106. In the first stage 3102, the display device 108 of the computer102 is illustrated as displaying first, second, third, and fourth images3108, 3110, 3112, 3114.

In the second stage 3104, the third image 3112 is illustrated as beingselected using a touch input, e.g., through use of a finger of theuser's hand 106, although other implementations are also contemplated. Astylus 116 is illustrated as providing a stylus input that describesmovement 3118 that begins within a boundary of the first image 3108,through the second image 3110, and ends at the third image 3112. Forinstance, the movement 3116 may involve placing the stylus 116 withinthe display of the first image 3108 and passing through the second image3110 to the third image 3112, where the stylus 116 is lifted away fromthe display device 108. From these inputs, the gesture module 104 mayrecognize the link gesture 140.

The link gesture 140 may be used to provide a variety of differentfunctionality. For example, the gesture module 104 may form links to beincluded with the third image 3112, an example of which is shown in thethird stage 3106. In this stage, a back 3118 of the image 3112 isillustrated that includes a display of metadata associated with theimage 3112, such as the title and type of the image. The metadata alsoincludes links to the first and second images 3108, 3110, which areillustrated as the titles taken from the images of “Mom,” and “Child.”The links are selectable to navigate to the respective images, e.g., thelink “Mom” is selectable to navigate to the first image 3108 and so on.Thus, the links may be formed using a simple gesture that does notinvolve manual input of text by a user. A variety of other functionalitymay also be made available via the link gesture 140, further discussionof which may be found in relation to FIGS. 32-33.

As previously described, although a specific implementation has beendescribed using touch and stylus inputs, it should be readily apparentthat a variety of other implementations are also contemplated. Forexample, the touch and stylus inputs may be switched to perform the linkgesture 140, the gesture may be performed using touch or stylus inputsalone, and so on. Additionally, the linking may be performed incombinations with a variety of different inputs. For instance, a pathmay be drawn around multiple objects, e.g., using a stylus to circle acollection, to select the objects within the path. An icon (e.g., agroup icon) may then be selected to link and/or group the objectstogether. A variety of other instances are also contemplated.

FIG. 32 is a flow diagram that depicts a procedure 3200 in an exampleimplementation of a link gesture in accordance with one or moreembodiments. Aspects of the procedure may be implemented in hardware,firmware, software, or a combination thereof. The procedure is shown asa set of blocks in this example that specify operations performed by oneor more devices and are not necessarily limited to the orders shown forperforming the operations by the respective blocks. In portions of thefollowing discussion, reference will be made to the environment 100 ofFIG. 1, the system 200 of FIG. 2, and the example implementation 3100 ofFIG. 31.

A first input is recognized as selecting an object displayed by adisplay device (block 3202), such as through selection using one or moretouch inputs, a stylus input, and so on. A second input is recognized asa line drawn from a second object displayed by the display device to thefirst object, the line recognized as being drawn while the first objectis selected (block 3204). For example, the line may be recognized asmovement 3116 of the stylus 116 from within a boundary of a secondobject (e.g., the second image 3112) to within a boundary of the objectthat is selected by the first input, e.g., the finger of the user's hand106 in the second stage 3104 of FIG. 31. Intervening image 3110, orother objects passed over by the stylus, may either be considered asadditional images that should also be linked together into a common set,or may be ignored as intermediary objects that are not the targets ofthe link gesture. The dynamics of the link gesture (e.g. inflectionpoints, momentary pauses while dragging, velocity thresholds, and so on)may be used to decide between these cases, if desired.

A link gesture is identified from the recognized first and secondinputs, the link gesture effective to create a link between the firstand second objects (block 3206). The gesture module 104, for instance,may identify the link gesture 140 and form a link that involves thefirst object selected by the first input and the second object that wasinvolved with the first object by the second input. The link may employa variety of functionality, such as a hyperlink to navigate between thefirst and second objects, store the link (e.g., with the first or secondobjects) for later navigation, provide an indication of the existence ofthe link (e.g., through underlining of the first or second object), andso on. A variety of other links are also contemplated, an example ofwhich may be found in relation to the following figure.

FIG. 33 is an illustration of another example implementation 3300 inwhich stages of the link gesture 140 of FIG. 1 are shown as being inputin conjunction with the computing device 102. The computing device 102is illustrated as outputting a user interface by a display device 108.The user interface includes a listing of playlists and a listing ofsongs.

A finger of the user's hand 3302 is illustrated as selecting a playlist“About Last Night” and the stylus 116 is illustrated as moved from thesong “My Way” to the selected playlist. In this way, metadata associatedwith the second object (e.g., the song) is associated with the selectedobject (e.g., the playlist), which in this case causes the song to beadded to the playlist. Thus, the gesture module 104 may identify thelink gesture 140 from the inputs and cause a corresponding operation tobe performed. Although formation of a playlist is described in thisexample, a variety of different metadata may be associated using thelink gesture 140, such as to classify movies by type, rate objects, andso on.

FIG. 34 is a flow diagram that depicts a procedure 3400 in an exampleimplementation of a link gesture in accordance with one or moreembodiments. Aspects of the procedure may be implemented in hardware,firmware, software, or a combination thereof. The procedure is shown asa set of blocks in this example that specify operations performed by oneor more devices and are not necessarily limited to the orders shown forperforming the operations by the respective blocks. In portions of thefollowing discussion, reference will be made to the environment 100 ofFIG. 1, the system 200 of FIG. 2, and the example implementation 3300 ofFIG. 33.

A first input is recognized as selecting an object displayed by adisplay device (block 3402). A second input is recognized as a linedrawn from a second object displayed by the display device to the firstobject, the line recognized as being drawn while the first object isselected (block 3404). For example, the line may be recognized as beingdrawn from a list of metadata to a song, a listing of places to animage, and so on.

A link gesture is identified from the recognized first and secondinputs, the link gesture effect to associate metadata represented by thesecond object with the first object (block 3406). Continuing with theprevious example, the link gesture 140 may be effective to cause themetadata to be stored as a part of the first object, e.g., so that theplaylist includes the song, the image includes a name of a person, andso on.

Yet again, it should be noted that although a specific example wasdescribed in which the link gesture 140 was input using touch and stylusinputs in FIGS. 31-34, those inputs may be switched, a single input type(e.g., touch or stylus) may be used to provide the inputs, and so on.

Contextual Spatial Multiplexing

FIG. 35 depicts an example implementation 3500 showing techniques forcontextual spatial multiplexing. In instances of the previous exampleimplementations, different types of inputs (e.g., stylus inputs versustouch inputs) were used to specify different gestures. For example, thebimodal input module 114 may be used to distinguish between the types ofinputs to identify gestures, such as one or more of the gesturespreviously described in relation to FIG. 1 and subsequent sections.

These techniques may also be leveraged for contextual spatialmultiplexing. Contextual spatial multiplexing describes techniques inwhich specific areas of a user interface take on different functions forstylus or touch inputs. For example, the finger of the user's hand 3502is illustrated as selecting the image 3504 at an initial point of theuser interface. Additionally, the stylus 116 is illustrated as writingthe word “Eleanor” 3506 which also began at the initial point in theuser interface. Thus, the bimodal input module 114 may distinguishbetween the types of inputs (e.g., touch versus stylus inputs) toprovide different functionality at the same point in the user interface.

In an implementation, primitives of touch (e.g., tap, hold, two-fingerhold, drag, cross, pinch, and other hand or finger postures) and stylus(e.g., tap, hold, drag-off, drag-into, cross, stroke) can be composed bythe bimodal input module 114 to create a larger possible space ofintuitive and semantically rich gestures than either stylus or touchalone. For instance, direct touch mode switching may integrate modeactivation, selection of object, and phrasing of subtasks into a singleobject-specific mode, e.g., to define a gesture as described above.

Additionally, the techniques may be composed, such as to arrive atdifferent gestures. For example, selecting an object along with thephrasing of subtasks together afford composing of multiple tools andeffects together. As previously described for the edge gesture 128 ofFIGS. 14-18, for instance, drawing and cutting using an edge of anobject were described. In other instances, a priority may be assigned tothe gestures by the gesture module to avoid potential ambiguities, e.g.cut takes precedence on edge gesture 128 that overlaps an item, ratherthan the brush gesture 132. Thus, in these implementations the styluswrites (or cuts) and touch manipulates, while the combination of stylusplus touch yields new techniques. But in some contexts other divisionsbetween stylus and touch are possible, and indeed consistent with userexpectations.

For example, the user interface displayed by the display device 108 ofthe computing device 102 may react differently depending on the area ofthe object that is engaged, and the context of surrounding objects andthe page (background). For example, ink notations on the user interfacemay be ignored for some touch inputs (e.g. selection, directmanipulation) to make it easier to perform two-finger zooming on thepage, as well as to avoid accidental disruption of stylus inputs, suchas ink strokes. The size of objects also may be considered, e.g. objectsthat are over a threshold size may be directly manipulated via touchinputs. A variety of other implementations are also contemplated,further discussion of which may be found in relation to the followingfigure.

FIG. 36 is a flow diagram that depicts a procedure 3600 in an exampleimplementation in which identification of whether an input is a stylusor touch input is used to identify an operation to be performed inconjunction with a user interface. Aspects of the procedure may beimplemented in hardware, firmware, software, or a combination thereof.The procedure is shown as a set of blocks in this example that specifyoperations performed by one or more devices and are not necessarilylimited to the orders shown for performing the operations by therespective blocks. In portions of the following discussion, referencewill be made to the environment 100 of FIG. 1, the system 200 of FIG. 2,and the example implementation 3500 of FIG. 35.

A determination is made as to whether an input is a touch input or astylus input, the input effective to indicate interaction with a userinterface displayed by a display device (block 3602). For example, thegesture module 104 may detect the input using a variety offunctionality, such as a touchscreen, cameras e.g., cameras includedalong with a plurality of pixels of the display device), and so on. Thegesture module 104 may then make a determination as to whether the inputis a touch input (e.g., entered using one or more fingers of a user'shand) or a stylus input (e.g., entered using a pointed input device).This determination may be performed in a variety of ways, such as bydetecting the stylus 116 using one or more sensors, based on an amountof the display device 108 that is contacted using the stylus versustouch, use of image recognition, and so on.

An operation is identified to be performed by a computing device basedat least in part on the determination such that the operation that isidentified differs based on whether the determined input is the touchinput or the stylus input (block 3604). The identified operation iscaused to be performed by the computing device (block 3606). As shown inFIG. 35, for example, the stylus input from the stylus 116 is used thewrite whereas the touch input from the finger of the user's hand 3502may be used to select and move the image 3504 from the same point withinthe user interface. A variety of other examples are also contemplated,such as based on configuration of an object at which the interaction isdirected. For instance, the gesture module 104 may be configured to makedistinctions on whether the object is an image, represents a song,pertains to a document, size of the object, and so on to enabledifferent operations to be performed based on the underlying and/ornearby objects. As another example, dragging a pen from a color pot mayleave a pen stroke, whereas dragging a finger from a color pot may leavea smudge or finger paint stroke. Selecting a color pot with the pen, andthen stroking with a finger; or conversely selecting a color pot with afinger, and then stroking the pen, may also imply different commands orparameters to commands (e.g. brush style, opacity, and so forth).Further discussion of such distinctions may be found in relation to thefollowing figure.

FIG. 37 is a flow diagram that depicts another procedure 3700 in anexample implementation in which identification of whether an input is astylus or touch input is used to identify an operation to be performedin conjunction with a user interface. Aspects of the procedure may beimplemented in hardware, firmware, software, or a combination thereof.The procedure is shown as a set of blocks in this example that specifyoperations performed by one or more devices and are not necessarilylimited to the orders shown for performing the operations by therespective blocks. In portions of the following discussion, referencewill be made to the environment 100 of FIG. 1, the system 200 of FIG. 2,and the example implementation 3500 of FIG. 35.

A determination is made that an input is a touch input or a stylusinput, the input effective to indicate interaction with a user interfacedisplayed by a display device (block 3702). This determination may beperformed in a variety of ways as previously and subsequently described.Responsive to the determination that the input is the touch input, afirst operation is caused to be performed in conjunction with the userinterface (block 3704). For example, the operation may involve moving anunderlying object, e.g., the image 3504 of FIG. 35.

Responsive to the determination that the input is the stylus input, asecond operation is caused to be performed in conjunction with the userinterface that is different than the first operation (block 3706).Continuing with the previous example, the stylus input provided by thestylus 116 may be used to write on the image 3504 as opposed to movingit. Additionally, it should be readily apparent that a variety of otherconsiderations may also be employed by the gesture module 104, such asproximity to other objects, “where” in the user interface theinteraction involving the input is to occur, and so on.

Example Device

FIG. 38 illustrates various components of an example device 3800 thatcan be implemented as any type of portable and/or computer device asdescribed with reference to FIGS. 1 and 2 to implement embodiments ofthe gesture techniques described herein. Device 3800 includescommunication devices 3802 that enable wired and/or wirelesscommunication of device data 3804 (e.g., received data, data that isbeing received, data scheduled for broadcast, data packets of the data,etc.). The device data 3804 or other device content can includeconfiguration settings of the device, media content stored on thedevice, and/or information associated with a user of the device. Mediacontent stored on device 3800 can include any type of audio, video,and/or image data. Device 3800 includes one or more data inputs 3806 viawhich any type of data, media content, and/or inputs can be received,such as user-selectable inputs, messages, music, television mediacontent, recorded video content, and any other type of audio, video,and/or image data received from any content and/or data source.

Device 3800 also includes communication interfaces 3808 that can beimplemented as any one or more of a serial and/or parallel interface, awireless interface, any type of network interface, a modem, and as anyother type of communication interface. The communication interfaces 3808provide a connection and/or communication links between device 3800 anda communication network by which other electronic, computing, andcommunication devices communicate data with device 3800.

Device 3800 includes one or more processors 3810 (e.g., any ofmicroprocessors, controllers, and the like) which process variouscomputer-executable instructions to control the operation of device 3800and to implement embodiments of a touch pull-in gesture. Alternativelyor in addition, device 3800 can be implemented with any one orcombination of hardware, firmware, or fixed logic circuitry that isimplemented in connection with processing and control circuits which aregenerally identified at 3812. Although not shown, device 3800 caninclude a system bus or data transfer system that couples the variouscomponents within the device. A system bus can include any one orcombination of different bus structures, such as a memory bus or memorycontroller, a peripheral bus, a universal serial bus, and/or a processoror local bus that utilizes any of a variety of bus architectures.

Device 3800 also includes computer-readable media 3814, such as one ormore memory components, examples of which include random access memory(RAM), non-volatile memory (e.g., any one or more of a read-only memory(ROM), flash memory, EPROM, EEPROM, etc.), and a disk storage device. Adisk storage device may be implemented as any type of magnetic oroptical storage device, such as a hard disk drive, a recordable and/orrewriteable compact disc (CD), any type of a digital versatile disc(DVD), and the like. Device 3800 can also include a mass storage mediadevice 3816.

Computer-readable media 3814 provides data storage mechanisms to storethe device data 3804, as well as various device applications 3818 andany other types of information and/or data related to operationalaspects of device 3800. For example, an operating system 3820 can bemaintained as a computer application with the computer-readable media3814 and executed on processors 3810. The device applications 3818 caninclude a device manager (e.g., a control application, softwareapplication, signal processing and control module, code that is nativeto a particular device, a hardware abstraction layer for a particulardevice, etc.). The device applications 3818 also include any systemcomponents or modules to implement embodiments of the gesture techniquesdescribed herein. In this example, the device applications 3818 includean interface application 3822 and a gesture-capture driver 3824 that areshown as software modules and/or computer applications. Thegesture-capture driver 3824 is representative of software that is usedto provide an interface with a device configured to capture a gesture,such as a touchscreen, track pad, camera, and so on. Alternatively or inaddition, the interface application 3822 and the gesture-capture driver3824 can be implemented as hardware, software, firmware, or anycombination thereof. Additionally, the gesture-capture driver 3824 maybe configured to support multiple input devices, such as separatedevices to capture touch and stylus inputs, respectively. For example,the device may be configured to include dual display devices, in whichone of the display device is configured to capture touch inputs whilethe other stylus inputs.

Device 3800 also includes an audio and/or video input-output system 3826that provides audio data to an audio system 3828 and/or provides videodata to a display system 3830. The audio system 3828 and/or the displaysystem 3830 can include any devices that process, display, and/orotherwise render audio, video, and image data. Video signals and audiosignals can be communicated from device 3800 to an audio device and/orto a display device via an RF (radio frequency) link, S-video link,composite video link, component video link, DVI (digital videointerface), analog audio connection, or other similar communicationlink. In an embodiment, the audio system 3828 and/or the display system3830 are implemented as external components to device 3800.Alternatively, the audio system 3828 and/or the display system 3830 areimplemented as integrated components of example device 3800.

CONCLUSION

Although the invention has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the invention defined in the appended claims is not necessarilylimited to the specific features or acts described. Rather, the specificfeatures and acts are disclosed as example forms of implementing theclaimed invention.

What is claimed is:
 1. A method performed on a computing device, the method comprising: recognizing a first touch input as selecting a first object displayed on a display device; recognizing a second touch input subsequent to the first input as a hold on a second object displayed on a display device; recognizing a third touch input as a tap on the second object during the hold on the second object; and identifying a staple gesture from the recognized first, second, and third touch inputs, the staple gesture effective to cause a collated display of the first object beneath the second object on the display device, where the collated display further comprises an indication that the first and second objects in the collated display are stapled together.
 2. The method of claim 1 where the first touch comprises a tap on the first object.
 3. The method of claim 1 where the hold on the second object is for at least a predetermined amount of time.
 4. The method of claim 1 further comprising recognizing a first number of additional touch inputs subsequent to the first touch input and prior to the second touch input as selecting a second number of additional objects displayed on the display device corresponding to first number, where the additional objects are each distinct from the first object and the second object and from each other of the additional objects.
 5. The method of claim 4 where the collated display further comprises the additional objects.
 6. The method of claim 5 where the additional objects are arranged in the collated display in an order in which the additional objects were selected.
 7. The method of claim 1 where further manipulation of the first and second objects in the collated display that are stapled together is as a group.
 8. A system comprising: a display device on which the system is configured to display a first object, a second object, and a third object; a touch screen on which the system is configured to: recognize a first touch input as selecting the first object displayed on the display device, recognize a second touch input subsequent to the first input as a hold on the second object displayed on the display device, and recognize a third touch input as a tap on the second object during the hold on the second object; and at least one processor via which the system is configured to identify a staple gesture from the recognized first, second, and third touch inputs, the staple gesture effective to cause a collated display of the first object beneath the second object on the display device, where the collated display further comprises an indication that the first and second objects in the collated display are stapled together.
 9. The system of claim 8 where the first touch comprises a tap on the first object.
 10. The system of claim 8 where the hold on the second object is for at least a predetermined amount of time.
 11. The system of claim 8, the touch screen on which the system is configured to recognize a first number of additional touch inputs subsequent to the first touch input and prior to the second touch input as selecting a second number of additional objects displayed on the display device corresponding to first number, where the additional objects are each distinct from the first object and the second object and from each other of the additional objects.
 12. The system of claim 11 where the collated display further comprises the additional objects.
 13. The system of claim 12 where the additional objects are arranged in the collated display in an order in which the additional objects were selected.
 14. The system of claim 8 where further manipulation of the first and second objects in the collated display that are stapled together is as a group.
 15. At least one hardware computer-readable medium that comprises computer-executable instructions that, based on execution by a computing device, configure the computing device to perform actions comprising: recognizing a first touch input as selecting a first object displayed on a display device; recognizing a second touch input subsequent to the first input as a hold on a second object displayed on the display device; recognizing a third touch input as a tap on the second object during the hold on the second object; and identifying a staple gesture from the recognized first, second, and third touch inputs, the staple gesture effective to cause a collated display of the first object beneath the second object on the display device, where the collated display further comprises an indication that the first and second objects in the collated display are stapled together.
 16. The at least one hardware computer-readable medium of claim 15 where the first touch comprises a tap on the first object, or where further manipulation of the first and second objects in the collated display are stapled together is as a group.
 17. The at least one hardware computer-readable medium of claim 15 where the hold on the second object is for at least a predetermined amount of time.
 18. The at least one hardware computer-readable medium of claim 15 further comprising recognizing a first number of additional touch inputs subsequent to the first touch input and prior to the second touch input as selecting a second number of additional objects displayed on the display device corresponding to first number, where the additional objects are each distinct from the first object and the second object and from each other of the additional objects.
 19. The at least one hardware computer-readable medium of claim 18 where the collated display further comprises the additional objects.
 20. The at least one hardware computer-readable medium of claim 19 where the additional objects are arranged in the collated display in an order in which the additional objects were selected. 